[Date Prev][Date Next][Thread Prev][Thread Next][Date Index][Thread Index]
[gnuastro-commits] master b1ea932 054/113: Separate Segment program now
|
From: |
Mohammad Akhlaghi |
|
Subject: |
[gnuastro-commits] master b1ea932 054/113: Separate Segment program now available in 3D |
|
Date: |
Fri, 16 Apr 2021 10:33:44 -0400 (EDT) |
branch: master
commit b1ea932d7ca5b38b3a693afeaab43ebd5a9e7c5d
Merge: e11098d 1c6b577
Author: Mohammad Akhlaghi <mohammad@akhlaghi.org>
Commit: Mohammad Akhlaghi <mohammad@akhlaghi.org>
Separate Segment program now available in 3D
All the major changes in the master branch regarding spinning off
segmentation from NoiseChisel into another program.
---
.gitignore | 4 +
Makefile.am | 15 +-
NEWS | 176 +-
README | 11 +-
THANKS | 2 +
bin/TEMPLATE/Makefile.am | 4 +-
bin/TEMPLATE/TEMPLATE.c | 4 +-
bin/TEMPLATE/TEMPLATE.h | 4 +-
bin/TEMPLATE/args.h | 4 +-
bin/TEMPLATE/authors-cite.h | 4 +-
bin/TEMPLATE/main.c | 8 +-
bin/TEMPLATE/main.h | 14 +-
bin/TEMPLATE/ui.c | 9 +-
bin/TEMPLATE/ui.h | 4 +-
bin/arithmetic/arithmetic.c | 505 ++-
bin/arithmetic/arithmetic.h | 8 +-
bin/arithmetic/main.c | 2 +-
bin/arithmetic/main.h | 1 -
bin/arithmetic/operands.c | 26 +-
bin/buildprog/Makefile.am | 36 +-
bin/convertt/Makefile.am | 4 +-
bin/convertt/convertt.c | 92 +-
bin/convertt/eps.c | 472 --
bin/convertt/eps.h | 41 -
bin/convertt/ui.c | 101 +-
bin/convolve/ui.c | 13 +-
bin/fits/args.h | 4 +-
bin/fits/fits.c | 13 +-
bin/fits/keywords.c | 76 +-
bin/fits/main.h | 4 +-
bin/mkcatalog/Makefile.am | 4 +-
bin/mkcatalog/args.h | 182 +-
bin/mkcatalog/astmkcatalog.conf | 7 +-
bin/mkcatalog/columns.c | 753 ++--
bin/mkcatalog/main.c | 2 +-
bin/mkcatalog/main.h | 54 +-
bin/mkcatalog/mkcatalog.c | 855 +---
bin/mkcatalog/mkcatalog.h | 15 +-
bin/mkcatalog/parse.c | 670 +++
bin/{convertt/jpeg.h => mkcatalog/parse.h} | 24 +-
bin/mkcatalog/ui.c | 1205 +++--
bin/mkcatalog/ui.h | 25 +-
bin/mkcatalog/upperlimit.c | 369 +-
bin/mkcatalog/upperlimit.h | 4 +
bin/mknoise/mknoise.c | 5 +-
bin/mknoise/ui.c | 6 +-
bin/mkprof/main.h | 2 +-
bin/mkprof/profiles.c | 3 +-
bin/mkprof/ui.c | 9 +-
bin/noisechisel/Makefile.am | 8 +-
bin/noisechisel/args.h | 249 +-
bin/noisechisel/astnoisechisel-3d.conf | 14 +-
bin/noisechisel/astnoisechisel.conf | 18 +-
bin/noisechisel/authors-cite.h | 2 +-
bin/noisechisel/detection.c | 70 +-
bin/noisechisel/detection.h | 2 +-
bin/noisechisel/kernel-3d.h | 50 -
bin/noisechisel/main.c | 2 +-
bin/noisechisel/main.h | 43 +-
bin/noisechisel/noisechisel.c | 186 +-
bin/noisechisel/noisechisel.h | 2 +-
bin/noisechisel/sky.c | 31 +-
bin/noisechisel/sky.h | 2 +-
bin/noisechisel/threshold.c | 49 +-
bin/noisechisel/threshold.h | 2 +-
bin/noisechisel/ui.c | 189 +-
bin/noisechisel/ui.h | 29 +-
bin/{TEMPLATE => segment}/Makefile.am | 10 +-
bin/{noisechisel => segment}/args.h | 538 +--
bin/segment/astsegment-3d.conf | 43 +
bin/segment/astsegment.conf | 41 +
bin/{noisechisel => segment}/authors-cite.h | 6 +-
bin/{noisechisel => segment}/clumps.c | 719 +--
bin/{noisechisel => segment}/clumps.h | 19 +-
bin/{noisechisel => segment}/main.c | 16 +-
bin/segment/main.h | 110 +
.../segmentation.c => segment/segment.c} | 365 +-
.../segmentation.h => segment/segment.h} | 10 +-
bin/segment/ui.c | 1051 +++++
bin/{noisechisel => segment}/ui.h | 68 +-
bin/statistics/sky.c | 2 +-
bin/statistics/ui.c | 11 +-
bin/table/ui.c | 51 +-
bin/warp/ui.c | 12 +-
bin/warp/warp.c | 69 +-
configure.ac | 138 +-
doc/Makefile.am | 18 +-
doc/announce-acknowledge.txt | 4 +
doc/gnuastro.texi | 4647 +++++++++++++-------
doc/release-checklist.txt | 19 +
lib/Makefile.am | 30 +-
lib/arithmetic-and.c | 2 +-
lib/arithmetic-or.c | 2 +-
lib/array.c | 174 +
lib/blank.c | 51 +
lib/data.c | 8 +-
lib/eps.c | 438 ++
lib/fits.c | 19 +-
lib/gnuastro-internal/commonopts.h | 2 +-
lib/gnuastro-internal/kernel-2d.h | 129 +
lib/gnuastro-internal/kernel-3d.h | 138 +
lib/gnuastro/array.h | 75 +
lib/gnuastro/binary.h | 2 +-
lib/gnuastro/blank.h | 3 +
lib/gnuastro/dimension.h | 1 +
lib/gnuastro/eps.h | 75 +
lib/gnuastro/fits.h | 6 +-
lib/gnuastro/jpeg.h | 75 +
lib/gnuastro/label.h | 74 +
bin/mkcatalog/upperlimit.h => lib/gnuastro/pdf.h | 55 +-
lib/gnuastro/statistics.h | 1 +
lib/gnuastro/tiff.h | 74 +
{bin/convertt => lib}/jpeg.c | 97 +-
lib/label.c | 529 +++
lib/list.c | 2 +-
lib/pdf.c | 125 +
lib/statistics.c | 675 +--
lib/tiff.c | 606 +++
lib/wcs.c | 4 +
tests/Makefile.am | 27 +-
.../{where.sh => connected-components.sh} | 7 +-
tests/arithmetic/or.sh | 4 +-
tests/arithmetic/snimage.sh | 8 +-
tests/arithmetic/where.sh | 2 +-
tests/buildprog/simpleio.c | 2 +-
tests/lib/multithread.c | 2 +-
tests/mkcatalog/aperturephot.sh | 6 +-
tests/mkcatalog/{aperturephot.sh => detections.sh} | 13 +-
tests/mkcatalog/{simple.sh => objects-clumps.sh} | 4 +-
tests/mkcatalog/simple-3d.sh | 4 +-
tests/noisechisel/noisechisel.sh | 8 +-
tests/prepconf.sh | 5 +-
.../{arithmetic/where.sh => segment/segment-3d.sh} | 9 +-
tests/{arithmetic/or.sh => segment/segment.sh} | 8 +-
134 files changed, 12103 insertions(+), 6248 deletions(-)
diff --git a/.gitignore b/.gitignore
index 9c71165..3db7e67 100644
--- a/.gitignore
+++ b/.gitignore
@@ -77,6 +77,7 @@ config.h
stamp-h1
Makefile
asttable
+astmatch
ltmain.sh
.dirstamp
configure
@@ -106,11 +107,14 @@ doc/stamp-vti
astarithmetic
aststatistics
astnoisechisel
+tests/simpleio
astsubtractsky
autom4te.cache
lib/gnuastro.pc
doc/gnuastro.cps
doc/gnuastro.vrs
+tests/versioncxx
+tests/multithread
lib/gnuastro/config.h
diff --git a/Makefile.am b/Makefile.am
index f62192f..d316836 100644
--- a/Makefile.am
+++ b/Makefile.am
@@ -83,6 +83,9 @@ endif
if COND_NOISECHISEL
MAYBE_NOISECHISEL = bin/noisechisel
endif
+if COND_SEGMENT
+ MAYBE_SEGMENT = bin/segment
+endif
if COND_STATISTICS
MAYBE_STATISTICS = bin/statistics
endif
@@ -110,12 +113,12 @@ endif
## conditions above). When `MAYBE_TEMPLATE' is not defined, then Make will
## see it as a blank string and igonore it, so there is no problem with
## having an uncommented `MAYBE_TEMPLATE' as a value in `SUBDIRS'.
-SUBDIRS = bootstrapped/lib $(MAYBE_GNULIBCHECK) lib $(MAYBE_ARITHMETIC) \
- $(MAYBE_BUILDPROG) $(MAYBE_CONVERTT) $(MAYBE_CONVOLVE) \
- $(MAYBE_COSMICCAL) $(MAYBE_CROP) $(MAYBE_FITS) $(MAYBE_MATCH) \
- $(MAYBE_MKCATALOG) $(MAYBE_MKNOISE) $(MAYBE_MKPROF) \
- $(MAYBE_NOISECHISEL) $(MAYBE_STATISTICS) $(MAYBE_TABLE) \
- $(MAYBE_TEMPLATE) $(MAYBE_WARP) doc tests
+SUBDIRS = bootstrapped/lib $(MAYBE_GNULIBCHECK) lib $(MAYBE_ARITHMETIC) \
+ $(MAYBE_BUILDPROG) $(MAYBE_CONVERTT) $(MAYBE_CONVOLVE) \
+ $(MAYBE_COSMICCAL) $(MAYBE_CROP) $(MAYBE_FITS) $(MAYBE_MATCH) \
+ $(MAYBE_MKCATALOG) $(MAYBE_MKNOISE) $(MAYBE_MKPROF) $(MAYBE_NOISECHISEL) \
+ $(MAYBE_SEGMENT) $(MAYBE_STATISTICS) $(MAYBE_TABLE) $(MAYBE_TEMPLATE) \
+ $(MAYBE_WARP) doc tests
diff --git a/NEWS b/NEWS
index 0fd4fd6..96ac855 100644
--- a/NEWS
+++ b/NEWS
@@ -5,48 +5,164 @@ GNU Astronomy Utilities NEWS -*-
outline -*-
** New features
- MakeCatalog: The new `--mean' and `--median' options will calculate the
- mean and median pixel value within an object or clump respectively.
-
- MakeCatalog: The new `--upperlimitsigma' and `--upperlimitquantile'
- columns will report the postion of the object's brightness with respect
- to the distribution of randomly measured values. The former as a multiple
- of sigma and the latter as a quantile. Also, `--upperlimitonesigma' will
- return the 1-sigma value of the randomly placed upper limit magnitudes
- (irrespective of the value given to `--upnsigma').
-
- NoiseChisel: a value of `none' to the `--kernel' option will disable
- convolution.
-
- Statistics: the new `--manualbinrange' allows the bins in histograms or
- cumulative frequency plots to be set outside the minimum or maximum
- values of the dataset.
+ All programs:
+ - Input image dataset(s) can be in any of the formats recognized by
+ Gnuastro (e.g., FITS, TIFF, JPEG), provided that their libraries
+ (optional dependencies) were present at installation time.
+
+ New program:
+ - Segment: new program in charge of segmentation over detections. This
+ operation was previously done by NoiseChisel. NoiseChisel is now ONLY
+ in charge of detection.
+
+ Arithmetic:
+ - connected-components: label the connected elements of the input.
+ - filter-sigclip-mean: sigma-clipped, mean filter operator.
+ - filter-sigclip-median: sigma-clipped, median filter operator.
+ - invert: subtract the maximum of unsigned types (absorption to emission).
+
+ ConvertType:
+ - TIFF images can also be used as input.
+
+ MakeCatalog:
+ MakeCatalog will only read the datasets necessary for the requested
+ columns. Until now, it would read all the possible datasets and all the
+ intermediate measurements. This is thus major improvement in memory and
+ CPU usage. As a result, the input argument is no longer assumed to be
+ the values file, but the object labels file. Please see the
+ "MakeCatalog inputs and basic settings" section of the book for
+ more. Here is the summary of the the options:
+ --insky: Sky value as a single value or file (dataset).
+ --instd: Sky standard deviation as a single value or file (dataset).
+ --valuesfile: filename containing the values dataset.
+ --valueshdu: HDU/extension containing the values dataset.
+ --clumpscat: Make a clumps catalog also (`WCLUMPS' keyword not used
anymore).
+ --noclumpsort: Don't sort the clumps catalog by host object ID.
+ --subtractsky: Subtract the given Sky from the values dataset.
+ --variance: input standard deviation image is actually variance.
+ --checkupperlimit: make a table for random positions and measurements.
+ --geoarea: the number of labeled pixels (irrespective of value).
+ --brightnesserr: error in estimating the brightness.
+ --mean: calculate the mean pixel value within an object or clump.
+ --median: calculate the median pixel value within an object or clump.
+ --upperlimitsigma: position in random distribution (in units of sigma).
+ --upperlimitquantile: position in random distribution (quantile).
+ --upperlimitonesigma: 1sigma value of the random distribution.
+ --upperlimitskew: (mean-median)/sigma or skewness of random distribution.
+
+ NoiseChisel:
+ --rawoutput: only output the detection labels and Sky and its STD.
+ --label: label the connected detections. Until now this was the default
+ behavior. However, from this release, NoiseChisel is only in charge
+ of detection. Segmentation is done by a new program (Segment). Since
+ detection is ultimately just a binary operator, the default output
+ now has an 8-bit unsigned integer type with values of 0 or 1. With
+ this option, you can ask for it to label/count the connected
+ detections instead of the default binary output.
+
+ Statistics:
+ --manualbinrange: histogram or CFP range can be outside of distribution.
+
+ Libraries:
+ gal_array_read: read array from any of known formats (FITS,TIFF,JPEG,...).
+ gal_array_read_to_type: similar to `gal_array_read', but to given type.
+ gal_array_read_one_ch: Read in the data and make sure it is in one channel.
+ gal_array_read_one_ch_to_type: Make sure input is in one channel and type.
+ gal_blank_is: check to see if argument is blank in its type or not.
+ gal_eps_name_is_eps: Returns 1 if given filename is EPS.
+ gal_eps_suffix_is_eps: Returns 1 if given suffix is EPS.
+ gal_eps_to_pt: Converts dataset size to PostScript points.
+ gal_eps_write: Writes a dataset into an EPS file.
+ gal_jpeg_name_is_jpeg: Returns 1 if given filename is JPEG.
+ gal_jpeg_suffix_is_jpeg: Returns 1 if given suffix is JPEG.
+ gal_jpeg_read: Reads input JPEG image into `gal_data_t'.
+ gal_jpeg_write: Writes a `gal_data_t' into a JPEG file.
+ gal_label_grow_indexs: grow known indexs into desired areas.
+ gal_label_oversegment: apply over-segmentation to an input dataset.
+ gal_pdf_name_is_pdf: Returns 1 if given filename is PDF.
+ gal_pdf_suffix_is_pdf: Returns 1 if given suffix is PDF.
+ gal_pdf_write: Writes a dataset into an PDF file.
+ gal_tiff_name_is_tiff: check if name contains a TIFF suffix.
+ gal_tiff_suffix_is_tiff: check if suffix is a TIFF suffix.
+ gal_tiff_dir_string_read: convert a string to a TIFF directory number.
+ gal_tiff_read: Read the contents of a TIFF "directory" to `gal_data_t'.
** Removed features
-** Changed features
-
- Libraries: `gal_statistics_quantile_function' returns `inf' or `-inf' if
- the given value is smaller than the minimum or larger than the maximum of
- the input dataset's range.
+ NoiseChisel:
-** Bug fixes
+ - Segmentation (and thus the options below) moved to the new Segment
+ program: --onlydetection, --segsnminarea, --checkclumpsn, --segquant,
+ --keepmaxnearriver, --gthresh, --minriverlength, --objbordersn,
+ --grownclumps, --checksegmentation.
- Many unused result warnings for asprintf in some compilers (bug #52979).
+ --skysubtracted: no longer necessary, included in noise measuremnts.
- Configure time CPPFLAGS and LDFLAGS don't pass to BuildProgram (bug #53122).
+ MakeCatalog:
+ --skysubtracted: no longer necessary, included in noise measuremnts.
- Crash when printing values with the `--onlyversion' option (bug #53142).
+** Changed features
- NULL value of onlyversion option causing a crash (bug #53147).
+ Fits:
+ --history: can be called/written multiple times in one run.
+ --comment: can be called/written multiple times in one run.
+
+ NoiseChisel:
+ From this release, NoiseChisel is only in charge of detection and won't
+ do segmentation any more. The new Segment program is in charge of
+ segmentation. Many of the changes below are due to this new limited
+ scope of NoiseChisel.
+ --kernel: value `none' will disable convolution.
+ - Renamed options:
+ --convolvedhdu ==> --chdu
+ --wkhdu ==> --whdu
+ --detsnminarea ==> --snminarea
+ --checkdetsn ==> --checksn
+ --detquant ==> --snquant
+ - By default the detection map is a binary image (values of 0 or 1).
+ - With no output name, the output has a `_detected.fits' suffix.
+
+ MakeCatalog:
+ - Estimation of noise-level is now done per-pixel over the whole
+ label. Until now the average noise level was used.
+ --objectsfile has been removed. The main input argument is now assumed
+ to be the objects file.
+
+ Table:
+ --column: multiple columns (comma separated) can be used in one
+ instance of this option (multiple instances of this option are
+ still acceptable also).
+
+ Libraries:
+ gal_fits_img_read: now only reads the data not the WCS, therefore it no
+ longer needs the last two arguments. A subsequent call to
+ `gal_wcs_read' can be used to read the WCS information in the file.
+
+ gal_statistics_quantile_function: returns `inf' or `-inf' if the given
+ value is smaller than the minimum or larger than the maximum of the
+ input dataset's range. Until now, it would return blank in such
+ cases.
+
+ gal_statistics_number: the output dataset now has a `size_t' type. Until
+ now it was `uint64_t'.
- Match output directory ignored when making multiple files (bug #53226).
+** Bug fixes
- Statistics program bad results on integer columns with limits (bug #53230).
+ bug #52979: Many unused result warnings for asprintf in some compilers.
+ bug #53122: Configure time CPPFLAGS and LDFLAGS don't pass to BuildProgram.
+ bug #53142: Crash when printing values with the `--onlyversion' option.
+ bug #53147: NULL value of onlyversion option causing a crash.
+ bug #53226: Match output directory ignored when making multiple files.
+ bug #53230: Statistics program bad results on integer columns with limits.
+ bug #53268: NoiseChisel crash when no growth is possible.
+ bug #53295: MakeCatalog parses area larger than clump.
+ bug #53304: NoiseChisel crash when there is no detection.
+ bug #53312: Fits crash on keyword editing (except --delete).
+ bug #53407: Instrumental noise in MakeNoise should be squared.
+ bug #53424: Sigma-clipping seg-faults when there are no more elements.
+ bug #53580: Warp crash when no WCS present.
- NoiseChisel crash when no growth is possible (bug #53268).
- MakeCatalog parses area larger than clump (bug #53295).
diff --git a/README b/README
index 45e258e..30c0247 100644
--- a/README
+++ b/README
@@ -76,10 +76,13 @@ context under categories/chapters.
Monte Carlo integration. It can also build the profiles on an
over-sampled image.
- - NoiseChisel (astnoisechisel): Detect and segment signal in noise. It
- uses a technique to detect very faint and diffuse, irregularly shaped
- signal in noise (galaxies in the sky), using thresholds that are below
- the Sky value (see arXiv:1505.01664).
+ - NoiseChisel (astnoisechisel): Detect and signal in noise. It uses a
+ technique to detect very faint and diffuse, irregularly shaped signal
+ in noise (galaxies in the sky), using thresholds that are below the Sky
+ value (see arXiv:1505.01664).
+
+ - Segment (astsegment): Segment a detection based on the structure of
+ signal within it.
- Statistics (aststatistics): Get pixel statistics and save histogram and
cumulative frequency plots.
diff --git a/THANKS b/THANKS
index 7ae2873..5103035 100644
--- a/THANKS
+++ b/THANKS
@@ -39,6 +39,7 @@ support in Gnuastro. The list is ordered alphabetically (by
family name).
Alan Lefor alefor@astr.tohoku.ac.jp
Guillaume Mahler guillaume.mahler@univ-lyon1.fr
Francesco Montanari francesco.montanari@openmailbox.org
+ Bertrand Pain bertrand.pain@inserm.fr
William Pence william.pence@nasa.gov
Bob Proulx bob@proulx.com
Yahya Sefidbakht y.sefidbakht@gmail.com
@@ -55,6 +56,7 @@ support in Gnuastro. The list is ordered alphabetically (by
family name).
Aaron Watkins aaron.watkins@oulu.fi
Christopher Willmer cnaw@as.arizona.edu
Sara Yousefi Taemeh s.yousefi.t@gmail.com
+ Johannes Zabl johannes.zabl@irap.omp.eu
Teams
diff --git a/bin/TEMPLATE/Makefile.am b/bin/TEMPLATE/Makefile.am
index f784618..8f159f2 100644
--- a/bin/TEMPLATE/Makefile.am
+++ b/bin/TEMPLATE/Makefile.am
@@ -1,9 +1,9 @@
## Process this file with automake to produce Makefile.inx
##
## Original author:
-## Your Name <your@email>
+## Mohammad akhlaghi <mohammad@akhlaghi.org>
## Contributing author(s):
-## Copyright (C) YYYY, Free Software Foundation, Inc.
+## Copyright (C) 2016-2018, Free Software Foundation, Inc.
##
## Gnuastro is free software: you can redistribute it and/or modify it
## under the terms of the GNU General Public License as published by
diff --git a/bin/TEMPLATE/TEMPLATE.c b/bin/TEMPLATE/TEMPLATE.c
index 60d6340..912b646 100644
--- a/bin/TEMPLATE/TEMPLATE.c
+++ b/bin/TEMPLATE/TEMPLATE.c
@@ -3,9 +3,9 @@ TEMPLATE - A minimal set of files and functions to define a
program.
TEMPLATE is part of GNU Astronomy Utilities (Gnuastro) package.
Original author:
- Your Name <your@email>
+ Mohammad akhlaghi <mohammad@akhlaghi.org>
Contributing author(s):
-Copyright (C) YYYY, Free Software Foundation, Inc.
+Copyright (C) 2016-2018, Free Software Foundation, Inc.
Gnuastro is free software: you can redistribute it and/or modify it
under the terms of the GNU General Public License as published by the
diff --git a/bin/TEMPLATE/TEMPLATE.h b/bin/TEMPLATE/TEMPLATE.h
index 8349e7a..5f5e662 100644
--- a/bin/TEMPLATE/TEMPLATE.h
+++ b/bin/TEMPLATE/TEMPLATE.h
@@ -3,9 +3,9 @@ TEMPLATE - A minimal set of files and functions to define a
program.
TEMPLATE is part of GNU Astronomy Utilities (Gnuastro) package.
Original author:
- Your Name <your@email>
+ Mohammad akhlaghi <mohammad@akhlaghi.org>
Contributing author(s):
-Copyright (C) YYYY, Free Software Foundation, Inc.
+Copyright (C) 2016-2018, Free Software Foundation, Inc.
Gnuastro is free software: you can redistribute it and/or modify it
under the terms of the GNU General Public License as published by the
diff --git a/bin/TEMPLATE/args.h b/bin/TEMPLATE/args.h
index 1b78e62..e4ae3eb 100644
--- a/bin/TEMPLATE/args.h
+++ b/bin/TEMPLATE/args.h
@@ -3,9 +3,9 @@ TEMPLATE - A minimal set of files and functions to define a
program.
TEMPLATE is part of GNU Astronomy Utilities (Gnuastro) package.
Original author:
- Your Name <your@email>
+ Mohammad akhlaghi <mohammad@akhlaghi.org>
Contributing author(s):
-Copyright (C) YYYY, Free Software Foundation, Inc.
+Copyright (C) 2016-2018, Free Software Foundation, Inc.
Gnuastro is free software: you can redistribute it and/or modify it
under the terms of the GNU General Public License as published by the
diff --git a/bin/TEMPLATE/authors-cite.h b/bin/TEMPLATE/authors-cite.h
index 8da944d..607abde 100644
--- a/bin/TEMPLATE/authors-cite.h
+++ b/bin/TEMPLATE/authors-cite.h
@@ -3,9 +3,9 @@ TEMPLATE - A minimal set of files and functions to define a
program.
TEMPLATE is part of GNU Astronomy Utilities (Gnuastro) package.
Original author:
- Your Name <your@email>
+ Mohammad akhlaghi <mohammad@akhlaghi.org>
Contributing author(s):
-Copyright (C) YYYY, Free Software Foundation, Inc.
+Copyright (C) 2016-2018, Free Software Foundation, Inc.
Gnuastro is free software: you can redistribute it and/or modify it
under the terms of the GNU General Public License as published by the
diff --git a/bin/TEMPLATE/main.c b/bin/TEMPLATE/main.c
index aff41a4..a7c450c 100644
--- a/bin/TEMPLATE/main.c
+++ b/bin/TEMPLATE/main.c
@@ -3,9 +3,9 @@ TEMPLATE - A minimal set of files and functions to define a
program.
TEMPLATE is part of GNU Astronomy Utilities (Gnuastro) package.
Original author:
- Your Name <your@email>
+ Mohammad akhlaghi <mohammad@akhlaghi.org>
Contributing author(s):
-Copyright (C) YYYY, Free Software Foundation, Inc.
+Copyright (C) 2016-2018, Free Software Foundation, Inc.
Gnuastro is free software: you can redistribute it and/or modify it
under the terms of the GNU General Public License as published by the
@@ -40,14 +40,14 @@ main (int argc, char *argv[])
struct timeval t1;
struct TEMPLATEparams p={{{0},0},0};
- /* Set they starting time. */
+ /* Set the starting time. */
time(&p.rawtime);
gettimeofday(&t1, NULL);
/* Read the input parameters. */
ui_read_check_inputs_setup(argc, argv, &p);
- /* Run MakeProfiles */
+ /* Run TEMPLATE */
TEMPLATE(&p);
/* Free all non-freed allocations. */
diff --git a/bin/TEMPLATE/main.h b/bin/TEMPLATE/main.h
index ef81396..9a46115 100644
--- a/bin/TEMPLATE/main.h
+++ b/bin/TEMPLATE/main.h
@@ -3,9 +3,9 @@ TEMPLATE - A minimal set of files and functions to define a
program.
TEMPLATE is part of GNU Astronomy Utilities (Gnuastro) package.
Original author:
- Your Name <your@email>
+ Mohammad akhlaghi <mohammad@akhlaghi.org>
Contributing author(s):
-Copyright (C) YYYY, Free Software Foundation, Inc.
+Copyright (C) 2016-2018, Free Software Foundation, Inc.
Gnuastro is free software: you can redistribute it and/or modify it
under the terms of the GNU General Public License as published by the
@@ -43,13 +43,13 @@ along with Gnuastro. If not, see
<http://www.gnu.org/licenses/>.
struct TEMPLATEparams
{
/* From command-line */
- struct gal_options_common_params cp; /* Common parameters. */
- gal_list_str_t *multivalue; /* Pointer to multivalue. */
- char *inputname; /* Input filename. */
- uint8_t *onoff; /* How to store on/off options. */
+ struct gal_options_common_params cp; /* Common parameters. */
+ char *inputname; /* Input filename. */
+ gal_list_str_t *multivalue; /* List of values given to "multivalue" */
+ uint8_t onoff; /* ==1 if option is called, ==0 otherwise. */
/* Output: */
- time_t rawtime; /* Starting time of the program. */
+ time_t rawtime; /* Starting time of the program. */
};
#endif
diff --git a/bin/TEMPLATE/ui.c b/bin/TEMPLATE/ui.c
index 8f11ebe..10a5886 100644
--- a/bin/TEMPLATE/ui.c
+++ b/bin/TEMPLATE/ui.c
@@ -3,9 +3,9 @@ TEMPLATE - A minimal set of files and functions to define a
program.
TEMPLATE is part of GNU Astronomy Utilities (Gnuastro) package.
Original author:
- Your Name <your@email>
+ Mohammad akhlaghi <mohammad@akhlaghi.org>
Contributing author(s):
-Copyright (C) YYYY, Free Software Foundation, Inc.
+Copyright (C) 2016-2018, Free Software Foundation, Inc.
Gnuastro is free software: you can redistribute it and/or modify it
under the terms of the GNU General Public License as published by the
@@ -219,12 +219,15 @@ ui_check_options_and_arguments(struct TEMPLATEparams *p)
a HDU is also given. */
if(p->inputname)
{
+ /* Check if it exists. */
+ gal_checkset_check_file(p->inputname);
+
+ /* If it is FITS, a HDU is also mandatory. */
if( gal_fits_name_is_fits(p->inputname) && p->cp.hdu==NULL )
error(EXIT_FAILURE, 0, "no HDU specified. When the input is a FITS "
"file, a HDU must also be specified, you can use the `--hdu' "
"(`-h') option and give it the HDU number (starting from "
"zero), extension name, or anything acceptable by CFITSIO");
-
}
else
error(EXIT_FAILURE, 0, "no input file is specified");
diff --git a/bin/TEMPLATE/ui.h b/bin/TEMPLATE/ui.h
index 290caa1..163bf42 100644
--- a/bin/TEMPLATE/ui.h
+++ b/bin/TEMPLATE/ui.h
@@ -3,9 +3,9 @@ TEMPLATE - A minimal set of files and functions to define a
program.
TEMPLATE is part of GNU Astronomy Utilities (Gnuastro) package.
Original author:
- Your Name <your@email>
+ Mohammad akhlaghi <mohammad@akhlaghi.org>
Contributing author(s):
-Copyright (C) YYYY, Free Software Foundation, Inc.
+Copyright (C) 2016-2018, Free Software Foundation, Inc.
Gnuastro is free software: you can redistribute it and/or modify it
under the terms of the GNU General Public License as published by the
diff --git a/bin/arithmetic/arithmetic.c b/bin/arithmetic/arithmetic.c
index dd560bd..baef282 100644
--- a/bin/arithmetic/arithmetic.c
+++ b/bin/arithmetic/arithmetic.c
@@ -31,6 +31,8 @@ along with Gnuastro. If not, see
<http://www.gnu.org/licenses/>.
#include <gnuastro/wcs.h>
#include <gnuastro/fits.h>
+#include <gnuastro/array.h>
+#include <gnuastro/binary.h>
#include <gnuastro/threads.h>
#include <gnuastro/dimension.h>
#include <gnuastro/statistics.h>
@@ -131,14 +133,16 @@ pop_number_of_operands(struct arithmeticparams *p,
gal_data_t *data,
struct arithmetic_filter_p
{
- int operator; /* The type of filtering. */
- size_t *fsize; /* Filter size. */
- size_t *hpfsize; /* Positive Half-filter size. */
- size_t *hnfsize; /* Negative Half-filter size. */
- gal_data_t *input; /* Input dataset. */
- gal_data_t *out; /* Output dataset. */
-
- int hasblank; /* If the dataset has blank values.*/
+ int operator; /* The type of filtering. */
+ size_t *fsize; /* Filter size. */
+ size_t *hpfsize; /* Positive Half-filter size. */
+ size_t *hnfsize; /* Negative Half-filter size. */
+ float sclip_multip; /* Sigma multiple in sigma-clipping. */
+ float sclip_param; /* Termination critera in sigma-cliping. */
+ gal_data_t *input; /* Input dataset. */
+ gal_data_t *out; /* Output dataset. */
+
+ int hasblank; /* If the dataset has blank values. */
};
@@ -153,9 +157,9 @@ arithmetic_filter(void *in_prm)
struct arithmetic_filter_p *afp=(struct arithmetic_filter_p *)tprm->params;
gal_data_t *input=afp->input;
- size_t ind,index;
- int out_type_checked=0;
- gal_data_t *result=NULL;
+ size_t sind=-1;
+ size_t ind, index, one=1;
+ gal_data_t *sigclip, *result=NULL;
size_t *hpfsize=afp->hpfsize, *hnfsize=afp->hnfsize;
size_t *tsize, *dsize=input->dsize, *fsize=afp->fsize;
size_t i, j, coord[ARITHMETIC_FILTER_DIM], ndim=input->ndim;
@@ -172,85 +176,128 @@ arithmetic_filter(void *in_prm)
/* Go over all the pixels that were assigned to this thread. */
for(i=0; tprm->indexs[i] != GAL_BLANK_SIZE_T; ++i)
{
- /* Get the coordinate of the pixel. */
+ /* For easy reading, put the index in `ind'. */
ind=tprm->indexs[i];
- gal_dimension_index_to_coord(ind, ndim, dsize, coord);
- /* See which dimensions need trimming. */
- tile->size=1;
- for(j=0;j<ndim;++j)
+ /* If we are on a blank element, then just set the output to blank
+ also. */
+ if( afp->hasblank
+ && gal_blank_is(gal_data_ptr_increment(input->array, ind,
+ input->type), input->type) )
+ gal_blank_write(gal_data_ptr_increment(afp->out->array, ind,
+ afp->out->type),
+ afp->out->type);
+ else
{
- /* Estimate the coordinate of the filter's starting point. Note
- that we are dealing with size_t (unsigned int) type here, so
- there are no negatives. A negative result will produce an
- extremely large number, so instead of checking for negative,
- we can just see if the result of a subtraction is less than
- the width of the input. */
- if( (coord[j] - hnfsize[j] > dsize[j])
- || (coord[j] + hpfsize[j] >= dsize[j]) )
+ /* Get the coordinate of the pixel. */
+ gal_dimension_index_to_coord(ind, ndim, dsize, coord);
+
+ /* See which dimensions need trimming. */
+ tile->size=1;
+ for(j=0;j<ndim;++j)
{
- start[j] = ( (coord[j] - hnfsize[j] > dsize[j])
- ? 0 : coord[j] - hnfsize[j] );
- end[j] = ( (coord[j] + hpfsize[j] >= dsize[j])
- ? dsize[j]
- : coord[j] + hpfsize[j] + 1);
- tsize[j] = end[j] - start[j];
+ /* Estimate the coordinate of the filter's starting
+ point. Note that we are dealing with size_t (unsigned int)
+ type here, so there are no negatives. A negative result
+ will produce an extremely large number, so instead of
+ checking for negative, we can just see if the result of a
+ subtraction is less than the width of the input. */
+ if( (coord[j] - hnfsize[j] > dsize[j])
+ || (coord[j] + hpfsize[j] >= dsize[j]) )
+ {
+ start[j] = ( (coord[j] - hnfsize[j] > dsize[j])
+ ? 0 : coord[j] - hnfsize[j] );
+ end[j] = ( (coord[j] + hpfsize[j] >= dsize[j])
+ ? dsize[j]
+ : coord[j] + hpfsize[j] + 1);
+ tsize[j] = end[j] - start[j];
+ }
+ else /* NOT on the edge (given requested filter width). */
+ {
+ tsize[j] = fsize[j];
+ start[j] = coord[j] - hnfsize[j];
+ }
+ tile->size *= tsize[j];
}
- else /* We are NOT on the edge (given requested filter width). */
+
+ /* For a test.
+ printf("coord: %zu, %zu\n", coord[1]+1, coord[0]+1);
+ printf("\tstart: %zu, %zu\n", start[1]+1, start[0]+1);
+ printf("\ttsize: %zu, %zu\n", tsize[1], tsize[0]);
+ */
+
+ /* Set the tile's starting pointer. */
+ index=gal_dimension_coord_to_index(ndim, dsize, start);
+ tile->array=gal_data_ptr_increment(input->array, index,
+ input->type);
+
+ /* Do the necessary calculation. */
+ switch(afp->operator)
{
- tsize[j] = fsize[j];
- start[j] = coord[j] - hnfsize[j];
- }
- tile->size *= tsize[j];
- }
+ case ARITHMETIC_OP_FILTER_MEDIAN:
+ result=gal_statistics_median(tile, 0);
+ break;
- /* Set the tile's starting pointer. */
- index=gal_dimension_coord_to_index(ndim, dsize, start);
- tile->array=gal_data_ptr_increment(input->array, index, input->type);
- /* Do the necessary calculation. */
- switch(afp->operator)
- {
- case ARITHMETIC_OP_FILTER_MEDIAN:
- result=gal_statistics_median(tile, 0);
- break;
+ case ARITHMETIC_OP_FILTER_MEAN:
+ result=gal_statistics_mean(tile);
+ break;
- case ARITHMETIC_OP_FILTER_MEAN:
- result=gal_statistics_mean(tile);
- break;
- default:
- error(EXIT_FAILURE, 0, "%s: a bug! Please contact us at %s to "
- "fix the problem. `afp->operator' code %d is not "
- "recognized", PACKAGE_BUGREPORT, __func__, afp->operator);
- }
+ case ARITHMETIC_OP_FILTER_SIGCLIP_MEAN:
+ case ARITHMETIC_OP_FILTER_SIGCLIP_MEDIAN:
+ /* Find the sigma-clipped results. */
+ sigclip=gal_statistics_sigma_clip(tile, afp->sclip_multip,
+ afp->sclip_param, 0, 1);
- /* Make sure the output array type and result's type are the same. We
- only need to do this once, but we'll suffice to once for each
- thread for simplicify of the code, it is too negligible to have
- any real affect. */
- if( out_type_checked==0)
- {
- if(result->type!=afp->out->type )
- error(EXIT_FAILURE, 0, "%s: a bug! please contact us at %s so "
- "we can address the problem. The tyes of `result' and "
- "`out' aren't the same, they are respectively: `%s' and "
- "`%s'", __func__, PACKAGE_BUGREPORT,
- gal_type_name(result->type, 1),
- gal_type_name(afp->out->type, 1));
- out_type_checked=1;
- }
+ /* Set the required index. */
+ switch(afp->operator)
+ {
+ case ARITHMETIC_OP_FILTER_SIGCLIP_MEAN: sind = 2; break;
+ case ARITHMETIC_OP_FILTER_SIGCLIP_MEDIAN: sind = 1; break;
+ default:
+ error(EXIT_FAILURE, 0, "%s: a bug! Please contact us at "
+ "%s to fix the problem. The `afp->operator' value "
+ "%d is not recognized as sigma-clipped median or "
+ "mean", __func__, PACKAGE_BUGREPORT, afp->operator);
+ }
+
+ /* Allocate the output and write the value into it. */
+ result=gal_data_alloc(NULL, GAL_TYPE_FLOAT32, 1, &one, NULL,
+ 0, -1, NULL, NULL, NULL);
+ ((float *)(result->array))[0] =
+ ((float *)(sigclip->array))[sind];
+
+ /* Clean up. */
+ gal_data_free(sigclip);
+ break;
+
+
+ default:
+ error(EXIT_FAILURE, 0, "%s: a bug! Please contact us at %s "
+ "to fix the problem. `afp->operator' code %d is not "
+ "recognized", PACKAGE_BUGREPORT, __func__,
+ afp->operator);
+ }
- /* Copy the result into the output array. */
- memcpy(gal_data_ptr_increment(afp->out->array, ind, afp->out->type),
- result->array, gal_type_sizeof(afp->out->type));
+ /* Make sure the output array type and result's type are the
+ same. */
+ if(result->type!=afp->out->type)
+ result=gal_data_copy_to_new_type_free(result, afp->out->type);
- /* Clean up. */
- gal_data_free(result);
+
+ /* Copy the result into the output array. */
+ memcpy(gal_data_ptr_increment(afp->out->array, ind,
+ afp->out->type),
+ result->array, gal_type_sizeof(afp->out->type));
+
+ /* Clean up for this pixel. */
+ gal_data_free(result);
+ }
}
- /* Clean up. */
+ /* Clean up for this thread. */
tile->array=NULL;
tile->block=NULL;
gal_data_free(tile);
@@ -268,12 +315,14 @@ arithmetic_filter(void *in_prm)
static void
wrapper_for_filter(struct arithmeticparams *p, char *token, int operator)
{
- size_t i=0, ndim, one=1;
int type=GAL_TYPE_INVALID;
+ size_t i=0, ndim, nparams, one=1;
struct arithmetic_filter_p afp={0};
size_t fsize[ARITHMETIC_FILTER_DIM];
- gal_data_t *tmp, *tmp2, *zero, *comp, *fsize_list=NULL;
+ gal_data_t *tmp, *tmp2, *zero, *comp, *params_list=NULL;
size_t hnfsize[ARITHMETIC_FILTER_DIM], hpfsize[ARITHMETIC_FILTER_DIM];
+ int issigclip=(operator==ARITHMETIC_OP_FILTER_SIGCLIP_MEAN
+ || operator==ARITHMETIC_OP_FILTER_SIGCLIP_MEDIAN);
/* Get the input's number of dimensions. */
@@ -297,21 +346,40 @@ wrapper_for_filter(struct arithmeticparams *p, char
*token, int operator)
NULL, NULL);
- /* Based on the dimensions of the popped operand, pop the necessary
- number of operands. */
- for(i=0;i<ndim;++i)
- gal_list_data_add(&fsize_list, operands_pop(p, token));
+ /* Based on the first popped operand's dimensions and the operator, of
+ pop the necessary number of operands. */
+ nparams = ndim + (issigclip ? 2 : 0 );
+ for(i=0;i<nparams;++i)
+ gal_list_data_add(¶ms_list, operands_pop(p, token));
- /* Make sure the filter size only has single values. */
+ /* Make sure the parameters only have single values. */
i=0;
- for(tmp=fsize_list; tmp!=NULL; tmp=tmp->next)
+ for(tmp=params_list; tmp!=NULL; tmp=tmp->next)
{
++i;
if(tmp->size!=1)
- error(EXIT_FAILURE, 0, "the filter length values given to the "
- "filter operators can only be numbers. Value number %zu has "
- "%zu elements, so its an array", ndim-i-1, tmp->size);
+ error(EXIT_FAILURE, 0, "the parameters given to the filtering "
+ "operators can only be numbers. Value number %zu has %zu "
+ "elements, so its an array", i, tmp->size);
+ }
+
+
+ /* If this is a sigma-clipping filter, the top two operands are the
+ sigma-clipping parameters. */
+ if(issigclip)
+ {
+ /* Read the sigma-clipping multiple (first element in the list). */
+ tmp=gal_list_data_pop(¶ms_list);
+ tmp=gal_data_copy_to_new_type_free(tmp, GAL_TYPE_FLOAT32);
+ afp.sclip_multip=*(float *)(tmp->array);
+ gal_data_free(tmp);
+
+ /* Read the sigma-clipping termination parameter. */
+ tmp=gal_list_data_pop(¶ms_list);
+ tmp=gal_data_copy_to_new_type_free(tmp, GAL_TYPE_FLOAT32);
+ afp.sclip_param=*(float *)(tmp->array);
+ gal_data_free(tmp);
}
@@ -324,7 +392,7 @@ wrapper_for_filter(struct arithmeticparams *p, char *token,
int operator)
values must be written in the inverse order since the user gives
dimensions with the FITS standard. */
i=ndim-1;
- for(tmp=fsize_list; tmp!=NULL; tmp=tmp->next)
+ for(tmp=params_list; tmp!=NULL; tmp=tmp->next)
{
/* Make sure the user has given an integer type. */
if(tmp->type==GAL_TYPE_FLOAT32 || tmp->type==GAL_TYPE_FLOAT64)
@@ -338,6 +406,7 @@ wrapper_for_filter(struct arithmeticparams *p, char *token,
int operator)
error(EXIT_FAILURE, 0, "lengths of filter along dimensions "
"must be positive. The given length in dimension %zu"
"is either zero or negative", ndim-i);
+ gal_data_free(comp);
/* Convert the input into size_t and put it into the array that
keeps the filter size. */
@@ -370,18 +439,26 @@ wrapper_for_filter(struct arithmeticparams *p, char
*token, int operator)
{ hnfsize[i]=fsize[i]/2; hpfsize[i]=fsize[i]/2-1; }
}
+ /* For a test.
+ printf("fsize: %zu, %zu\n", fsize[0], fsize[1]);
+ printf("hnfsize: %zu, %zu\n", hnfsize[0], hnfsize[1]);
+ printf("hpfsize: %zu, %zu\n", hpfsize[0], hpfsize[1]);
+ */
/* See if the input has blank pixels. */
afp.hasblank=gal_blank_present(afp.input, 1);
+
/* Set the type of the output dataset. */
switch(operator)
{
case ARITHMETIC_OP_FILTER_MEDIAN:
+ case ARITHMETIC_OP_FILTER_SIGCLIP_MEDIAN:
type=afp.input->type;
break;
case ARITHMETIC_OP_FILTER_MEAN:
+ case ARITHMETIC_OP_FILTER_SIGCLIP_MEAN:
type=GAL_TYPE_FLOAT64;
break;
@@ -391,12 +468,14 @@ wrapper_for_filter(struct arithmeticparams *p, char
*token, int operator)
PACKAGE_BUGREPORT, __func__, operator);
}
+
/* Allocate the output dataset. Note that filtering doesn't change
the units of the dataset. */
afp.out=gal_data_alloc(NULL, type, ndim, afp.input->dsize,
afp.input->wcs, 0, afp.input->minmapsize,
NULL, afp.input->unit, NULL);
+
/* Spin off threads for each pixel. */
gal_threads_spin_off(arithmetic_filter, &afp, afp.input->size,
p->cp.numthreads);
@@ -406,10 +485,115 @@ wrapper_for_filter(struct arithmeticparams *p, char
*token, int operator)
/* Add the output to the top of the stack. */
operands_add(p, NULL, afp.out);
-
/* Clean up and add the output on top of the stack */
+ gal_data_free(zero);
gal_data_free(afp.input);
- gal_list_data_free(fsize_list);
+ gal_list_data_free(params_list);
+}
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+/***************************************************************/
+/************* Other functions *************/
+/***************************************************************/
+static void
+arithmetic_connected_components(struct arithmeticparams *p, char *token)
+{
+ int conn_int;
+ gal_data_t *out=NULL;
+
+ /* Pop the two necessary operands. */
+ gal_data_t *conn = operands_pop(p, token);
+ gal_data_t *in = operands_pop(p, token);
+
+ /* Do proper sanity checks on `conn'. */
+ if(conn->size!=1)
+ error(EXIT_FAILURE, 0, "the first popped operand to "
+ "`connected-components' must be a single number. However, it has "
+ "%zu elements", conn->size);
+ if(conn->type==GAL_TYPE_FLOAT32 || conn->type==GAL_TYPE_FLOAT64)
+ error(EXIT_FAILURE, 0, "the first popped operand to "
+ "`connected-components' is the connectivity (a value between 1 "
+ "and the number of dimensions) therefore, it must NOT be a "
+ "floating point");
+
+ /* Convert the connectivity value to a 32-bit integer and read it in and
+ make sure it is not larger than the number of dimensions. */
+ conn=gal_data_copy_to_new_type_free(conn, GAL_TYPE_INT32);
+ conn_int = *((int32_t *)(conn->array));
+ if(conn_int>in->ndim)
+ error(EXIT_FAILURE, 0, "the first popped operand of "
+ "`connected-components' (%d) is larger than the number of "
+ "dimensions in the second-popped operand (%zu)", conn_int,
+ in->ndim);
+
+ /* Make sure the array has an unsigned 8-bit type. */
+ if(in->type!=GAL_TYPE_UINT8)
+ error(EXIT_FAILURE, 0, "the second popped operand of "
+ "`connected-components' doesn't have an 8-bit unsigned "
+ "integer type. It must be a binary dataset (only being equal "
+ "to zero is checked). You can use the `uint8' operator to "
+ "convert the type of this operand.");
+
+ /* Do the connected components labeling. */
+ gal_binary_connected_components(in, &out, 1);
+
+ /* Push the result onto the stack. */
+ operands_add(p, NULL, out);
+
+ /* Clean up. */
+ gal_data_free(in);
+ gal_data_free(conn);
+}
+
+
+
+
+
+static void
+arithmetic_invert(struct arithmeticparams *p, char *token)
+{
+ gal_data_t *in = operands_pop(p, token);
+
+ uint8_t *u8 = in->array, *u8f = u8 + in->size;
+ uint8_t *u16 = in->array, *u16f = u16 + in->size;
+ uint8_t *u32 = in->array, *u32f = u32 + in->size;
+ uint8_t *u64 = in->array, *u64f = u64 + in->size;
+
+ /* Do the inversion based on type. */
+ switch(in->type)
+ {
+ case GAL_TYPE_UINT8: do *u8 = UINT8_MAX-*u8; while(++u8<u8f); break;
+ case GAL_TYPE_UINT16: do *u16 = UINT16_MAX-*u16; while(++u16<u16f); break;
+ case GAL_TYPE_UINT32: do *u32 = UINT32_MAX-*u32; while(++u32<u32f); break;
+ case GAL_TYPE_UINT64: do *u64 = UINT64_MAX-*u64; while(++u64<u64f); break;
+ default:
+ error(EXIT_FAILURE, 0, "`invert' operand has %s type. `invert' can "
+ "only take unsigned integer types.\n\nYou can use any of the "
+ "`uint8', `uint16', `uint32', or `uint64' operators to chage "
+ "the type before calling `invert'",
+ gal_type_name(in->type, 1));
+ }
+
+ /* Push the result onto the stack. */
+ operands_add(p, NULL, in);
}
@@ -431,6 +615,7 @@ wrapper_for_filter(struct arithmeticparams *p, char *token,
int operator)
+
/***************************************************************/
/************* Reverse Polish algorithm *************/
/***************************************************************/
@@ -453,7 +638,7 @@ reversepolish(struct arithmeticparams *p)
/* Prepare the processing: */
p->operands=NULL;
- p->addcounter=p->popcounter=0;
+ p->popcounter=0;
/* Go over each input token and do the work. */
@@ -462,7 +647,7 @@ reversepolish(struct arithmeticparams *p)
/* If we have a name or number, then add it to the operands linked
list. Otherwise, pull out two members and do the specified
operation on them. */
- if(gal_fits_name_is_fits(token->v))
+ if(gal_array_name_recognized(token->v))
operands_add(p, token->v, NULL);
else if( (d1=gal_data_copy_string_to_number(token->v)) )
operands_add(p, NULL, d1);
@@ -472,130 +657,138 @@ reversepolish(struct arithmeticparams *p)
/* Order is the same as in the manual. */
/* Simple arithmetic operators. */
if (!strcmp(token->v, "+" ))
- { op=GAL_ARITHMETIC_OP_PLUS; nop=2; }
+ { op=GAL_ARITHMETIC_OP_PLUS; nop=2; }
else if (!strcmp(token->v, "-" ))
- { op=GAL_ARITHMETIC_OP_MINUS; nop=2; }
+ { op=GAL_ARITHMETIC_OP_MINUS; nop=2; }
else if (!strcmp(token->v, "x" ))
- { op=GAL_ARITHMETIC_OP_MULTIPLY; nop=2; }
+ { op=GAL_ARITHMETIC_OP_MULTIPLY; nop=2; }
else if (!strcmp(token->v, "/" ))
- { op=GAL_ARITHMETIC_OP_DIVIDE; nop=2; }
+ { op=GAL_ARITHMETIC_OP_DIVIDE; nop=2; }
else if (!strcmp(token->v, "%" ))
- { op=GAL_ARITHMETIC_OP_MODULO; nop=2; }
+ { op=GAL_ARITHMETIC_OP_MODULO; nop=2; }
/* Mathematical Operators. */
else if (!strcmp(token->v, "abs"))
- { op=GAL_ARITHMETIC_OP_ABS; nop=1; }
+ { op=GAL_ARITHMETIC_OP_ABS; nop=1; }
else if (!strcmp(token->v, "pow"))
- { op=GAL_ARITHMETIC_OP_POW; nop=2; }
+ { op=GAL_ARITHMETIC_OP_POW; nop=2; }
else if (!strcmp(token->v, "sqrt"))
- { op=GAL_ARITHMETIC_OP_SQRT; nop=1; }
+ { op=GAL_ARITHMETIC_OP_SQRT; nop=1; }
else if (!strcmp(token->v, "log"))
- { op=GAL_ARITHMETIC_OP_LOG; nop=1; }
+ { op=GAL_ARITHMETIC_OP_LOG; nop=1; }
else if (!strcmp(token->v, "log10"))
- { op=GAL_ARITHMETIC_OP_LOG10; nop=1; }
+ { op=GAL_ARITHMETIC_OP_LOG10; nop=1; }
/* Statistical/higher-level operators. */
else if (!strcmp(token->v, "minvalue"))
- { op=GAL_ARITHMETIC_OP_MINVAL; nop=1; }
+ { op=GAL_ARITHMETIC_OP_MINVAL; nop=1; }
else if (!strcmp(token->v, "maxvalue"))
- { op=GAL_ARITHMETIC_OP_MAXVAL; nop=1; }
+ { op=GAL_ARITHMETIC_OP_MAXVAL; nop=1; }
else if (!strcmp(token->v, "numvalue"))
- { op=GAL_ARITHMETIC_OP_NUMVAL; nop=1; }
+ { op=GAL_ARITHMETIC_OP_NUMVAL; nop=1; }
else if (!strcmp(token->v, "sumvalue"))
- { op=GAL_ARITHMETIC_OP_SUMVAL; nop=1; }
+ { op=GAL_ARITHMETIC_OP_SUMVAL; nop=1; }
else if (!strcmp(token->v, "meanvalue"))
- { op=GAL_ARITHMETIC_OP_MEANVAL; nop=1; }
+ { op=GAL_ARITHMETIC_OP_MEANVAL; nop=1; }
else if (!strcmp(token->v, "stdvalue"))
- { op=GAL_ARITHMETIC_OP_STDVAL; nop=1; }
+ { op=GAL_ARITHMETIC_OP_STDVAL; nop=1; }
else if (!strcmp(token->v, "medianvalue"))
- { op=GAL_ARITHMETIC_OP_MEDIANVAL; nop=1; }
+ { op=GAL_ARITHMETIC_OP_MEDIANVAL; nop=1; }
else if (!strcmp(token->v, "min"))
- { op=GAL_ARITHMETIC_OP_MIN; nop=-1; }
+ { op=GAL_ARITHMETIC_OP_MIN; nop=-1; }
else if (!strcmp(token->v, "max"))
- { op=GAL_ARITHMETIC_OP_MAX; nop=-1; }
+ { op=GAL_ARITHMETIC_OP_MAX; nop=-1; }
else if (!strcmp(token->v, "num"))
- { op=GAL_ARITHMETIC_OP_NUM; nop=-1; }
+ { op=GAL_ARITHMETIC_OP_NUM; nop=-1; }
else if (!strcmp(token->v, "sum"))
- { op=GAL_ARITHMETIC_OP_SUM; nop=-1; }
+ { op=GAL_ARITHMETIC_OP_SUM; nop=-1; }
else if (!strcmp(token->v, "mean"))
- { op=GAL_ARITHMETIC_OP_MEAN; nop=-1; }
+ { op=GAL_ARITHMETIC_OP_MEAN; nop=-1; }
else if (!strcmp(token->v, "std"))
- { op=GAL_ARITHMETIC_OP_STD; nop=-1; }
+ { op=GAL_ARITHMETIC_OP_STD; nop=-1; }
else if (!strcmp(token->v, "median"))
- { op=GAL_ARITHMETIC_OP_MEDIAN; nop=-1; }
+ { op=GAL_ARITHMETIC_OP_MEDIAN; nop=-1; }
/* Conditional operators. */
else if (!strcmp(token->v, "lt" ))
- { op=GAL_ARITHMETIC_OP_LT; nop=2; }
+ { op=GAL_ARITHMETIC_OP_LT; nop=2; }
else if (!strcmp(token->v, "le"))
- { op=GAL_ARITHMETIC_OP_LE; nop=2; }
+ { op=GAL_ARITHMETIC_OP_LE; nop=2; }
else if (!strcmp(token->v, "gt" ))
- { op=GAL_ARITHMETIC_OP_GT; nop=2; }
+ { op=GAL_ARITHMETIC_OP_GT; nop=2; }
else if (!strcmp(token->v, "ge"))
- { op=GAL_ARITHMETIC_OP_GE; nop=2; }
+ { op=GAL_ARITHMETIC_OP_GE; nop=2; }
else if (!strcmp(token->v, "eq"))
- { op=GAL_ARITHMETIC_OP_EQ; nop=2; }
+ { op=GAL_ARITHMETIC_OP_EQ; nop=2; }
else if (!strcmp(token->v, "ne"))
- { op=GAL_ARITHMETIC_OP_NE; nop=2; }
+ { op=GAL_ARITHMETIC_OP_NE; nop=2; }
else if (!strcmp(token->v, "and"))
- { op=GAL_ARITHMETIC_OP_AND; nop=2; }
+ { op=GAL_ARITHMETIC_OP_AND; nop=2; }
else if (!strcmp(token->v, "or"))
- { op=GAL_ARITHMETIC_OP_OR; nop=2; }
+ { op=GAL_ARITHMETIC_OP_OR; nop=2; }
else if (!strcmp(token->v, "not"))
- { op=GAL_ARITHMETIC_OP_NOT; nop=1; }
+ { op=GAL_ARITHMETIC_OP_NOT; nop=1; }
else if (!strcmp(token->v, "isblank"))
- { op=GAL_ARITHMETIC_OP_ISBLANK; nop=1; }
+ { op=GAL_ARITHMETIC_OP_ISBLANK; nop=1; }
else if (!strcmp(token->v, "where"))
- { op=GAL_ARITHMETIC_OP_WHERE; nop=3; }
+ { op=GAL_ARITHMETIC_OP_WHERE; nop=3; }
/* Bitwise operators. */
else if (!strcmp(token->v, "bitand"))
- { op=GAL_ARITHMETIC_OP_BITAND; nop=2; }
+ { op=GAL_ARITHMETIC_OP_BITAND; nop=2; }
else if (!strcmp(token->v, "bitor"))
- { op=GAL_ARITHMETIC_OP_BITOR; nop=2; }
+ { op=GAL_ARITHMETIC_OP_BITOR; nop=2; }
else if (!strcmp(token->v, "bitxor"))
- { op=GAL_ARITHMETIC_OP_BITXOR; nop=2; }
+ { op=GAL_ARITHMETIC_OP_BITXOR; nop=2; }
else if (!strcmp(token->v, "lshift"))
- { op=GAL_ARITHMETIC_OP_BITLSH; nop=2; }
+ { op=GAL_ARITHMETIC_OP_BITLSH; nop=2; }
else if (!strcmp(token->v, "rshift"))
- { op=GAL_ARITHMETIC_OP_BITRSH; nop=2; }
+ { op=GAL_ARITHMETIC_OP_BITRSH; nop=2; }
else if (!strcmp(token->v, "bitnot"))
- { op=GAL_ARITHMETIC_OP_BITNOT; nop=1; }
+ { op=GAL_ARITHMETIC_OP_BITNOT; nop=1; }
/* Type conversion. */
else if (!strcmp(token->v, "uint8"))
- { op=GAL_ARITHMETIC_OP_TO_UINT8; nop=1; }
+ { op=GAL_ARITHMETIC_OP_TO_UINT8; nop=1; }
else if (!strcmp(token->v, "int8"))
- { op=GAL_ARITHMETIC_OP_TO_INT8; nop=1; }
+ { op=GAL_ARITHMETIC_OP_TO_INT8; nop=1; }
else if (!strcmp(token->v, "uint16"))
- { op=GAL_ARITHMETIC_OP_TO_UINT16; nop=1; }
+ { op=GAL_ARITHMETIC_OP_TO_UINT16; nop=1; }
else if (!strcmp(token->v, "int16"))
- { op=GAL_ARITHMETIC_OP_TO_INT16; nop=1; }
+ { op=GAL_ARITHMETIC_OP_TO_INT16; nop=1; }
else if (!strcmp(token->v, "uint32"))
- { op=GAL_ARITHMETIC_OP_TO_UINT32; nop=1; }
+ { op=GAL_ARITHMETIC_OP_TO_UINT32; nop=1; }
else if (!strcmp(token->v, "int32"))
- { op=GAL_ARITHMETIC_OP_TO_INT32; nop=1; }
+ { op=GAL_ARITHMETIC_OP_TO_INT32; nop=1; }
else if (!strcmp(token->v, "uint64"))
- { op=GAL_ARITHMETIC_OP_TO_UINT64; nop=1; }
+ { op=GAL_ARITHMETIC_OP_TO_UINT64; nop=1; }
else if (!strcmp(token->v, "int64"))
- { op=GAL_ARITHMETIC_OP_TO_INT64; nop=1; }
+ { op=GAL_ARITHMETIC_OP_TO_INT64; nop=1; }
else if (!strcmp(token->v, "float32"))
- { op=GAL_ARITHMETIC_OP_TO_FLOAT32; nop=1; }
+ { op=GAL_ARITHMETIC_OP_TO_FLOAT32; nop=1; }
else if (!strcmp(token->v, "float64"))
- { op=GAL_ARITHMETIC_OP_TO_FLOAT64; nop=1; }
+ { op=GAL_ARITHMETIC_OP_TO_FLOAT64; nop=1; }
/* Filters. */
- else if (!strcmp(token->v, "filter-median"))
- { op=ARITHMETIC_OP_FILTER_MEDIAN; nop=0; }
else if (!strcmp(token->v, "filter-mean"))
- { op=ARITHMETIC_OP_FILTER_MEAN; nop=0; }
+ { op=ARITHMETIC_OP_FILTER_MEAN; nop=0; }
+ else if (!strcmp(token->v, "filter-median"))
+ { op=ARITHMETIC_OP_FILTER_MEDIAN; nop=0; }
+ else if (!strcmp(token->v, "filter-sigclip-mean"))
+ { op=ARITHMETIC_OP_FILTER_SIGCLIP_MEAN; nop=0; }
+ else if (!strcmp(token->v, "filter-sigclip-median"))
+ { op=ARITHMETIC_OP_FILTER_SIGCLIP_MEDIAN; nop=0; }
+ else if (!strcmp(token->v, "connected-components"))
+ { op=ARITHMETIC_OP_CONNECTED_COMPONENTS; nop=0; }
+ else if (!strcmp(token->v, "invert"))
+ { op=ARITHMETIC_OP_INVERT; nop=0; }
/* Finished checks with known operators */
else
error(EXIT_FAILURE, 0, "the argument \"%s\" could not be "
- "interpretted as a FITS file, number, or operator",
- token->v);
+ "interpretted as a recognized input file name, number, or "
+ "operator", token->v);
/* See if the arithmetic library must be called or not. */
@@ -658,9 +851,19 @@ reversepolish(struct arithmeticparams *p)
{
case ARITHMETIC_OP_FILTER_MEAN:
case ARITHMETIC_OP_FILTER_MEDIAN:
+ case ARITHMETIC_OP_FILTER_SIGCLIP_MEAN:
+ case ARITHMETIC_OP_FILTER_SIGCLIP_MEDIAN:
wrapper_for_filter(p, token->v, op);
break;
+ case ARITHMETIC_OP_CONNECTED_COMPONENTS:
+ arithmetic_connected_components(p, token->v);
+ break;
+
+ case ARITHMETIC_OP_INVERT:
+ arithmetic_invert(p, token->v);
+ break;
+
default:
error(EXIT_FAILURE, 0, "%s: a bug! please contact us at "
"%s to fix the problem. The code %d is not "
@@ -690,11 +893,9 @@ reversepolish(struct arithmeticparams *p)
filename=p->operands->filename;
if( gal_fits_name_is_fits(filename) )
{
- p->operands->data=gal_fits_img_read(filename,hdu,p->cp.minmapsize,
- 0, 0);
- p->refdata.wcs=p->operands->data->wcs;
- p->refdata.nwcs=p->operands->data->nwcs;
- p->operands->data->wcs=NULL;
+ p->operands->data=gal_array_read_one_ch(filename, hdu,
+ p->cp.minmapsize);
+ p->refdata.wcs=gal_wcs_read(filename, hdu, 0, 0, &p->refdata.nwcs);
if(!p->cp.quiet) printf(" - %s (hdu %s) is read.\n", filename, hdu);
}
else
@@ -762,7 +963,7 @@ reversepolish(struct arithmeticparams *p)
/************* Top function *************/
/***************************************************************/
void
-imgarith(struct arithmeticparams *p)
+arithmetic(struct arithmeticparams *p)
{
/* Parse the arguments */
reversepolish(p);
diff --git a/bin/arithmetic/arithmetic.h b/bin/arithmetic/arithmetic.h
index fc5b65c..59b2170 100644
--- a/bin/arithmetic/arithmetic.h
+++ b/bin/arithmetic/arithmetic.h
@@ -31,14 +31,18 @@ along with Gnuastro. If not, see
<http://www.gnu.org/licenses/>.
library. */
enum arithmetic_prog_operators
{
- ARITHMETIC_OP_FILTER_MEDIAN=GAL_ARITHMETIC_OP_LAST_CODE,
+ ARITHMETIC_OP_FILTER_MEDIAN = GAL_ARITHMETIC_OP_LAST_CODE,
ARITHMETIC_OP_FILTER_MEAN,
+ ARITHMETIC_OP_FILTER_SIGCLIP_MEAN,
+ ARITHMETIC_OP_FILTER_SIGCLIP_MEDIAN,
+ ARITHMETIC_OP_CONNECTED_COMPONENTS,
+ ARITHMETIC_OP_INVERT,
};
void
-imgarith(struct arithmeticparams *p);
+arithmetic(struct arithmeticparams *p);
#endif
diff --git a/bin/arithmetic/main.c b/bin/arithmetic/main.c
index fd21c48..eddbc57 100644
--- a/bin/arithmetic/main.c
+++ b/bin/arithmetic/main.c
@@ -46,7 +46,7 @@ main (int argc, char *argv[])
ui_read_check_inputs_setup(argc, argv, &p);
/* Run MakeProfiles */
- imgarith(&p);
+ arithmetic(&p);
/* Free any allocated space */
freeandreport(&p, &t1);
diff --git a/bin/arithmetic/main.h b/bin/arithmetic/main.h
index 29c2515..c0c2a1d 100644
--- a/bin/arithmetic/main.h
+++ b/bin/arithmetic/main.h
@@ -70,7 +70,6 @@ struct arithmeticparams
/* Input: */
gal_list_str_t *hdus; /* List of all given HDU strings. */
gal_list_str_t *tokens; /* List of all arithmetic tokens. */
- size_t addcounter; /* The number of FITS images added. */
size_t popcounter; /* The number of FITS images popped. */
gal_data_t refdata; /* Container for information of the data. */
char *globalhdu; /* Single HDU for all inputs. */
diff --git a/bin/arithmetic/operands.c b/bin/arithmetic/operands.c
index 5948bbe..48d3632 100644
--- a/bin/arithmetic/operands.c
+++ b/bin/arithmetic/operands.c
@@ -31,6 +31,8 @@ along with Gnuastro. If not, see
<http://www.gnu.org/licenses/>.
#include <gnuastro/wcs.h>
#include <gnuastro/fits.h>
+#include <gnuastro/tiff.h>
+#include <gnuastro/array.h>
#include <gnuastro-internal/checkset.h>
#include "main.h"
@@ -77,17 +79,18 @@ operands_add(struct arithmeticparams *p, char *filename,
gal_data_t *data)
newnode->data=data;
newnode->filename=filename;
- if(filename != NULL && gal_fits_name_is_fits(filename))
+ /* See if a HDU must be read or not. */
+ if(filename != NULL
+ && ( gal_fits_name_is_fits(filename)
+ || gal_tiff_name_is_tiff(filename) ) )
{
/* Set the HDU for this filename. */
if(p->globalhdu)
gal_checkset_allocate_copy(p->globalhdu, &newnode->hdu);
else
newnode->hdu=gal_list_str_pop(&p->hdus);
-
- /* Increment the FITS counter. */
- ++p->addcounter;
}
+ else newnode->hdu=NULL;
/* Make the link to the previous list. */
newnode->next=p->operands;
@@ -122,21 +125,12 @@ operands_pop(struct arithmeticparams *p, char *operator)
filename=operands->filename;
/* Read the dataset. */
- data=gal_fits_img_read(filename, hdu, p->cp.minmapsize, 0, 0);
+ data=gal_array_read_one_ch(filename, hdu, p->cp.minmapsize);
/* In case this is the first image that is read, then keep the WCS
- information in the `refdata' structure. Otherwise, the WCS is not
- necessary and we can safely free it. In any case, `data' must not
- have a WCS structure. */
+ information in the `refdata' structure. */
if(p->popcounter==0)
- {
- p->refdata.wcs=data->wcs;
- p->refdata.nwcs=data->nwcs;
- }
- else
- wcsfree(data->wcs);
- data->wcs=NULL;
- data->nwcs=0;
+ p->refdata.wcs=gal_wcs_read(filename, hdu, 0, 0, &p->refdata.nwcs);
/* When the reference data structure's dimensionality is non-zero, it
means that this is not the first image read. So, write its basic
diff --git a/bin/buildprog/Makefile.am b/bin/buildprog/Makefile.am
index 1c89124..b76dd2e 100644
--- a/bin/buildprog/Makefile.am
+++ b/bin/buildprog/Makefile.am
@@ -47,37 +47,35 @@ EXTRA_DIST = main.h authors-cite.h args.h ui.h buildprog.h
astbuildprog.conf.in
# CPPFLAGS and LDFLAGS that the user might have given at configure time
# (stored in `IN_CPPFLAGS' and `IN_LDFLAGS' Makefile
# variables). BuildProgram also needs to know these directories in order to
-# compile its programs. Here, we will be going
+# compile its programs. Thus with the rule below, the directories in these
+# variables will be written into the final BuildProgram configuration file.
astbuildprog.conf: $(top_srcdir)/bin/buildprog/astbuildprog.conf.in
cp $< $@
infoadded="no"; \
for i in $(IN_CPPFLAGS); do \
- if test x"$$i" != x; then \
- if test $$infoadded = "no"; then \
- echo "" >> $@; \
- echo "# Installation information" >> $@; \
- infoadded="yes"; \
- fi; \
- v=$$(echo $$i | sed -e 's/-I//'); \
- echo " includedir $$v" >> $@; \
- fi; \
+ if test $$infoadded = "no"; then \
+ echo "" >> $@; \
+ echo "# Installation information" >> $@; \
+ infoadded="yes"; \
+ fi; \
+ v=$$(echo $$i | sed -e 's/-I//'); \
+ echo " includedir $$v" >> $@; \
done; \
for i in $(IN_LDFLAGS); do \
- if test x"$$i" != x; then \
- if test $$infoadded = "no"; then \
- echo "" >> $@; \
- echo "# Installation information" >> $@; \
- infoadded="yes"; \
- fi; \
- v=$$(echo $$i | sed -e 's/-L//'); \
- echo " linkdir $$v" >> $@; \
+ if test $$infoadded = "no"; then \
+ echo "" >> $@; \
+ echo "# Installation information" >> $@; \
+ infoadded="yes"; \
fi; \
+ v=$$(echo $$i | sed -e 's/-L//'); \
+ echo " linkdir $$v" >> $@; \
done
-## The configuration file (distribute and install).
+## The configuration file (clean, distribute and install).
## NOTE: the man page is created in doc/Makefile.am
+CLEANFILES = astbuildprog.conf
dist_sysconf_DATA = astbuildprog.conf
diff --git a/bin/convertt/Makefile.am b/bin/convertt/Makefile.am
index 0b851f6..847d8ba 100644
--- a/bin/convertt/Makefile.am
+++ b/bin/convertt/Makefile.am
@@ -30,9 +30,9 @@ bin_PROGRAMS = astconvertt
astconvertt_LDADD = -lgnuastro
-astconvertt_SOURCES = main.c ui.c convertt.c eps.c jpeg.c
+astconvertt_SOURCES = main.c ui.c convertt.c
-EXTRA_DIST = main.h authors-cite.h args.h ui.h convertt.h eps.h jpeg.h
+EXTRA_DIST = main.h authors-cite.h args.h ui.h convertt.h
diff --git a/bin/convertt/convertt.c b/bin/convertt/convertt.c
index 72b1d54..7ed7ebf 100644
--- a/bin/convertt/convertt.c
+++ b/bin/convertt/convertt.c
@@ -29,8 +29,11 @@ along with Gnuastro. If not, see
<http://www.gnu.org/licenses/>.
#include <string.h>
#include <stdlib.h>
+#include <gnuastro/eps.h>
+#include <gnuastro/pdf.h>
#include <gnuastro/txt.h>
#include <gnuastro/fits.h>
+#include <gnuastro/jpeg.h>
#include <gnuastro/arithmetic.h>
#include <gnuastro-internal/timing.h>
@@ -38,9 +41,6 @@ along with Gnuastro. If not, see
<http://www.gnu.org/licenses/>.
#include "main.h"
-#include "eps.h"
-#include "jpeg.h"
-
@@ -165,57 +165,6 @@ convertt_truncate(struct converttparams *p)
/**************************************************************/
-/************** Save text and FITS ***************/
-/**************************************************************/
-static void
-save_with_gnuastro_lib(struct converttparams *p)
-{
- gal_data_t *channel;
-
- /* Determine the type. */
- switch(p->outformat)
- {
-
- /* FITS: a FITS file can have many extensions (channels). */
- case OUT_FORMAT_FITS:
- for(channel=p->chll; channel!=NULL; channel=channel->next)
- gal_fits_img_write(channel, p->cp.output, NULL, PROGRAM_NAME);
- break;
-
- /* Plain text: only one channel is acceptable. */
- case OUT_FORMAT_TXT:
- gal_checkset_writable_remove(p->cp.output, 0, p->cp.dontdelete);
- gal_txt_write(p->chll, NULL, p->cp.output);
- break;
-
-
- /* Not recognized. */
- default:
- error(EXIT_FAILURE, 0, "%s: a bug! output format code `%d' not "
- "recognized", __func__, p->outformat);
- }
-}
-
-
-
-
-
-
-
-
-
-
-
-
-
-
-
-
-
-
-
-
-/**************************************************************/
/************** convert to 8 bit ***************/
/**************************************************************/
void
@@ -361,6 +310,7 @@ convertt_scale_to_uchar(struct converttparams *p)
void
convertt(struct converttparams *p)
{
+ gal_data_t *channel;
/* Make any of the desired changes to the data. */
if(p->changeaftertrunc)
@@ -374,35 +324,41 @@ convertt(struct converttparams *p)
convertt_truncate(p);
}
-
-
/* Save the outputs: */
switch(p->outformat)
{
- case OUT_FORMAT_TXT:
+ /* FITS: a FITS file can have many extensions (channels). */
case OUT_FORMAT_FITS:
- save_with_gnuastro_lib(p);
+ for(channel=p->chll; channel!=NULL; channel=channel->next)
+ gal_fits_img_write(channel, p->cp.output, NULL, PROGRAM_NAME);
+ break;
+
+ /* Plain text: only one channel is acceptable. */
+ case OUT_FORMAT_TXT:
+ gal_checkset_writable_remove(p->cp.output, 0, p->cp.dontdelete);
+ gal_txt_write(p->chll, NULL, p->cp.output);
break;
+ /* JPEG: */
case OUT_FORMAT_JPEG:
-#ifdef HAVE_LIBJPEG
convertt_scale_to_uchar(p);
- jpeg_write(p);
-#else
- error(EXIT_FAILURE, 0, "you have asked for a JPEG output, however, "
- "when %s was configured libjpeg was not available. To write "
- "to JPEG files, libjpeg is required. Please install it and "
- "configure, make and install %s again", PACKAGE_STRING,
- PACKAGE_STRING);
-#endif
+ gal_jpeg_write(p->chll, p->cp.output, p->quality, p->widthincm);
break;
+ /* EPS. */
case OUT_FORMAT_EPS:
+ convertt_scale_to_uchar(p);
+ gal_eps_write(p->chll, p->cp.output, p->widthincm, p->borderwidth,
+ p->hex, 0);
+ break;
+
+ /* PDF */
case OUT_FORMAT_PDF:
convertt_scale_to_uchar(p);
- eps_write_eps_or_pdf(p);
+ gal_pdf_write(p->chll, p->cp.output, p->widthincm, p->borderwidth);
break;
+ /* Not recognized. */
default:
error(EXIT_FAILURE, 0, "%s: a bug! Please contact us so we can find "
"the problem and fix it The internal type of the output is "
diff --git a/bin/convertt/eps.c b/bin/convertt/eps.c
deleted file mode 100644
index 699f240..0000000
--- a/bin/convertt/eps.c
+++ /dev/null
@@ -1,472 +0,0 @@
-/*********************************************************************
-ConvertType - Convert between various types of files.
-ConvertType is part of GNU Astronomy Utilities (Gnuastro) package.
-
-Original author:
- Mohammad Akhlaghi <mohammad@akhlaghi.org>
-Contributing author(s):
-Copyright (C) 2015-2018, Free Software Foundation, Inc.
-
-Gnuastro is free software: you can redistribute it and/or modify it
-under the terms of the GNU General Public License as published by the
-Free Software Foundation, either version 3 of the License, or (at your
-option) any later version.
-
-Gnuastro is distributed in the hope that it will be useful, but
-WITHOUT ANY WARRANTY; without even the implied warranty of
-MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the GNU
-General Public License for more details.
-
-You should have received a copy of the GNU General Public License
-along with Gnuastro. If not, see <http://www.gnu.org/licenses/>.
-**********************************************************************/
-#include <config.h>
-
-#include <stdio.h>
-#include <errno.h>
-#include <error.h>
-#include <stdlib.h>
-#include <string.h>
-
-#include <gnuastro-internal/timing.h>
-#include <gnuastro-internal/checkset.h>
-
-#include "main.h"
-
-
-
-
-
-
-/*************************************************************
- ************** Acceptable EPS names ***************
- *************************************************************/
-int
-nameiseps(char *name)
-{
- size_t len;
- len=strlen(name);
- if ( ( len>=3 && strcmp(&name[len-3], "eps") == 0 )
- || ( len>=3 && strcmp(&name[len-3], "EPS") == 0 )
- || ( len>=4 && strcmp(&name[len-4], "epsf") == 0 )
- || ( len>=4 && strcmp(&name[len-4], "epsi") == 0 ) )
- return 1;
- else
- return 0;
-}
-
-
-
-
-
-int
-nameisepssuffix(char *name)
-{
- if (strcmp(name, "eps") == 0 || strcmp(name, ".eps") == 0
- || strcmp(name, "EPS") == 0 || strcmp(name, ".EPS") == 0
- || strcmp(name, "epsf") == 0 || strcmp(name, ".epsf") == 0
- || strcmp(name, "epsi") == 0 || strcmp(name, ".epsi") == 0)
- return 1;
- else
- return 0;
-}
-
-
-
-
-
-int
-nameispdf(char *name)
-{
- size_t len;
- len=strlen(name);
- if (strcmp(&name[len-3], "pdf") == 0
- || strcmp(&name[len-3], "PDF") == 0)
- return 1;
- else
- return 0;
-}
-
-
-
-
-
-int
-nameispdfsuffix(char *name)
-{
- if (strcmp(name, "pdf") == 0 || strcmp(name, ".pdf") == 0
- || strcmp(name, "PDF") == 0 || strcmp(name, ".PDF") == 0)
- return 1;
- else
- return 0;
-}
-
-
-
-
-
-
-
-
-
-
-
-
-
-
-
-
-
-
-
-/*************************************************************
- ************** Write an EPS image **************
- *************************************************************/
-static int
-eps_is_binary(struct converttparams *p)
-{
- gal_data_t *channel;
- unsigned char *i, *fi;
-
- /* Go through all the channels. */
- for(channel=p->chll; channel!=NULL; channel=channel->next)
- {
- /* Go through all the values and see they are 0 and break out of the
- loop as soon as you get to a pixel that is not 0 or `maxbyte'. */
- fi = (i=p->chll->array) + p->chll->size;
- do
- if(*i!=p->maxbyte && *i!=0) break;
- while(++i<fi);
-
- /* If we didn't get to the end of the channel, then we have a
- non-binary image. */
- if(i!=fi)
- return 0;
- }
-
- /* If we get to this point, then all the channels were binary, so return
- success. */
- return 1;
-}
-
-
-
-
-
-/* Show the bit values in a uint8_t variable. It is included here as a
- test in debugging problems with blackandwhite. To test it use a
- very small valued input to make the outputs reasonable. For example
- I am now testing it with an input text array of 12 elements (while
- calling the --invert option):
-
- 1 0 1 0 0 0
- 0 0 0 1 0 1
-*/
-void
-eps_show_bits(uint8_t x)
-{
- int i;
-
- for(i=7;i>=0;--i)
- (x&(1<<i)) ? putchar('1') : putchar('0');
- putchar('\n');
-}
-
-
-
-
-
-
-/* Convert the channels into into a 0 and 1 bit stream. This function is
- only called when the image is binary (has only two values). NOTE: each
- row has to have an integer number of bytes, so when the number of pixels
- in a row is not a multiple of 8, we'll add one. */
-size_t
-eps_convert_to_bitstream(struct converttparams *p)
-{
- gal_data_t *channel;
- size_t i, j, k, bytesinrow, bytesinimg;
- unsigned char *bits, byte, curbit, *in;
- size_t s0=p->chll->dsize[0], s1=p->chll->dsize[1];
-
- /* Find the size values and allocate the array. */
- if( s1 % 8 ) bytesinrow = s1/8 + 1;
- else bytesinrow = s1/8;
- bytesinimg = bytesinrow*s0;
-
- /* Go over all the channels. */
- for(channel=p->chll; channel!=NULL; channel=channel->next)
- {
- /* Allocate the array. */
- bits=gal_data_malloc_array(GAL_TYPE_UINT8, bytesinimg, __func__,
- "bits");
-
- /* Put the values in. */
- in=channel->array;
- for(i=0;i<s0;++i)
- {
- for(j=0;j<bytesinrow;++j)
- { /* i*s0+j is the byte, not bit position. */
- byte=0; /* Set the 8 bits to zero. */
- curbit=0x80; /* Current bit position, starting at: */
- for(k=0;k<8;++k)
- {
- if( j*8+k < s1 )
- {
- if(in[i*s1+j*8+k])
- byte |= curbit;
- curbit >>= 1;
- }
- else break;
- }
- /*eps_show_bits(byte);*/
- bits[i*bytesinrow+j]=byte;
- }
- }
- free(channel->array);
- channel->array=bits;
- channel->type=GAL_TYPE_BIT;
- }
-
- /* Return the total number of bytes in the image. */
- return bytesinimg;
-}
-
-
-
-
-
-void
-eps_write_hex(struct converttparams *p, FILE *fp, size_t size)
-{
- unsigned char *in;
- gal_data_t *channel;
- size_t i=0, j, elem_for_newline=35;
-
- for(channel=p->chll; channel!=NULL; channel=channel->next)
- {
- if(channel->status) /* A blank channel has status==1. */
- fprintf(fp, "{<00>} %% Channel %zu is blank\n", i);
- else
- {
- in=channel->array;
- fprintf(fp, "{<");
- for(j=0;j<size;++j)
- {
- fprintf(fp, "%02X", in[j]);
- if(j%elem_for_newline==0) fprintf(fp, "\n");
- }
- fprintf(fp, ">}\n");
- }
- ++i;
- }
-}
-
-
-
-
-
-void
-eps_write_ascii85(struct converttparams *p, FILE *fp, size_t size)
-{
- unsigned char *in;
- gal_data_t *channel;
- uint32_t anint, base;
- size_t i=0, j, k, elem_for_newline=15; /* 15*5=75 */
-
- for(channel=p->chll; channel!=NULL; channel=channel->next)
- {
- if(channel->status)
- fprintf(fp, "{<00>} %% Channel %zu is blank\n", i);
- else
- {
- in=channel->array;
- fprintf(fp, "{<~");
- for(j=0;j<size;j+=4)
- {
- /* This is the last four bytes */
- if(size-j<4)
- {
- anint=in[j]*256*256*256;
- if(size-j>1) anint+=in[j+1]*256*256;
- if(size-j==3) anint+=in[j+2]*256;
- }
- else
- anint=( in[j]*256*256*256 + in[j+1]*256*256
- + in[j+2]*256 + in[j+3] );
-
- /* If all four bytes are zero, then just print `z'. */
- if(anint==0) fprintf(fp, "z");
- else
- {
- /* To check, just change the fprintf below to printf:
- printf("\n\n");
- printf("%u %u %u %u\n", in[i], in[i+1],
- in[i+2], in[i+3]);
- */
- base=85*85*85*85;
- /* Do the ASCII85 encoding: */
- for(k=0;k<5;++k)
- {
- fprintf(fp, "%c", anint/base+33);
- anint%=base;
- base/=85;
- }
- }
- /* Go to the next line if on the right place: */
- if(j%elem_for_newline==0) fprintf(fp, "\n");
- }
- fprintf(fp, "~>}\n");
- }
- ++i;
- }
-}
-
-
-
-
-
-static void
-eps_write_image(struct converttparams *p, FILE *fp)
-{
- int bpc=8;
- size_t i, size, *dsize=p->chll->dsize;
-
- /* Set the number of bits per component. */
- if( p->numch==1 && eps_is_binary(p) )
- {
- bpc=1;
- size=eps_convert_to_bitstream(p);
- }
- else size=p->chll->size;
-
- switch(p->numch)
- {
- case 1: fprintf(fp, "/DeviceGray setcolorspace\n"); break;
- case 3: fprintf(fp, "/DeviceRGB setcolorspace\n"); break;
- case 4: fprintf(fp, "/DeviceCMYK setcolorspace\n"); break;
- default:
- error(EXIT_FAILURE, 0, "%s: a bug! The number of channels (%zu) is not "
- "1, 3 or 4. Please contact us so we can find the issue and fix it",
- __func__, p->numch);
- }
- fprintf(fp, "<<\n");
- fprintf(fp, " /ImageType 1\n");
- fprintf(fp, " /Width %zu\n", dsize[1]);
- fprintf(fp, " /Height %zu\n", dsize[0]);
- fprintf(fp, " /ImageMatrix [ %zu 0 0 %zu 0 0 ]\n", dsize[1], dsize[0]);
- fprintf(fp, " /MultipleDataSources true\n");
- fprintf(fp, " /BitsPerComponent %d\n", bpc);
- fprintf(fp, " /Decode[");
- for(i=0;i<p->numch;++i) {fprintf(fp, " 0 1");} fprintf(fp, " ]\n");
- fprintf(fp, " /Interpolate false\n");
- fprintf(fp, " /DataSource [\n");
- if(p->hex) eps_write_hex(p, fp, size);
- else eps_write_ascii85(p, fp, size);
- fprintf(fp, " ]\n");
- fprintf(fp, ">>\n");
- fprintf(fp, "image\n\n");
-}
-
-
-
-
-void
-eps_write_eps_or_pdf(struct converttparams *p)
-{
- FILE *fp;
- float hbw;
- char command[20000], *epsfilename=NULL;
- size_t winpt, hinpt, *dsize=p->chll->dsize;
-
-
- /* Find the bounding box */
- winpt=p->widthincm*72.0f/2.54f;
- hinpt=(float)( dsize[0] * winpt )/(float)(dsize[1]);
- hbw=(float)p->borderwidth/2.0f;
-
-
- /* EPS filename */
- if(p->outformat==OUT_FORMAT_EPS)
- {
- epsfilename=p->cp.output;
- gal_checkset_writable_remove(epsfilename, 0, p->cp.dontdelete);
- }
- else if (p->outformat==OUT_FORMAT_PDF)
- {
- gal_checkset_writable_remove(p->cp.output, 0, p->cp.dontdelete);
- epsfilename=gal_checkset_automatic_output(&p->cp, p->cp.output, ".ps");
- }
- else
- error(EXIT_FAILURE, 0, "%s: a bug! code %d not recognized for "
- "`p->outformat'", __func__, p->outformat);
-
-
- /* Open the output file and write the top comments. */
- errno=0;
- fp=fopen(epsfilename, "w");
- if(fp==NULL)
- error(EXIT_FAILURE, errno, "%s", p->cp.output);
- fprintf(fp, "%%!PS-Adobe-3.0 EPSF-3.0\n");
- fprintf(fp, "%%%%BoundingBox: 0 0 %zu %zu\n", winpt+2*p->borderwidth,
- hinpt+2*p->borderwidth);
- fprintf(fp, "%%%%Creator: %s\n", PROGRAM_STRING);
- fprintf(fp, "%%%%CreationDate: %s", ctime(&p->rawtime));
- fprintf(fp, "%%%%LanuageLevel: 3\n");
- fprintf(fp, "%%%%EndComments\n\n");
- if(p->outformat==OUT_FORMAT_EPS)
- fprintf(fp, "gsave\n\n");
-
-
- /* Commands to draw the border: */
- if(p->borderwidth)
- {
- fprintf(fp, "%% Draw the border:\n");
- fprintf(fp, "0 setgray\n");
- fprintf(fp, "%d setlinewidth\n", p->borderwidth);
- fprintf(fp, "%.1f %.1f moveto\n", hbw, hbw);
- fprintf(fp, "0 %zu rlineto\n", hinpt+p->borderwidth);
- fprintf(fp, "%zu 0 rlineto\n", winpt+p->borderwidth);
- fprintf(fp, "0 -%zu rlineto\n", hinpt+p->borderwidth);
- fprintf(fp, "closepath\n");
- fprintf(fp, "stroke\n\n");
- }
-
-
-
- /* Write the image: */
- fprintf(fp, "%% Draw the image:\n");
- fprintf(fp, "%d %d translate\n", p->borderwidth, p->borderwidth);
- fprintf(fp, "%zu %zu scale\n", winpt, hinpt);
- eps_write_image(p, fp);
-
-
-
- /* Ending of the EPS file: */
- if(p->outformat==OUT_FORMAT_EPS)
- fprintf(fp, "grestore\n");
- else
- fprintf(fp, "showpage\n");
- fprintf(fp, "%%%%EOF");
- fclose(fp);
-
-
-
- if(p->outformat==OUT_FORMAT_PDF)
- {
- sprintf(command, "gs -q -o %s -sDEVICE=pdfwrite -dDEVICEWIDTHPOINTS=%zu"
- " -dDEVICEHEIGHTPOINTS=%zu -dPDFFitPage %s", p->cp.output,
- winpt+2*p->borderwidth, hinpt+2*p->borderwidth, epsfilename);
- if(system(command))
- error(EXIT_FAILURE, 0, "the command to convert a PostScript file to "
- "PDF (`%s') was not successful! The PostScript file (%s) is "
- "left if you want to convert or use it through any other "
- "means", command, epsfilename);
- sprintf(command, "rm %s", epsfilename);
- if(system(command))
- error(EXIT_FAILURE, 0, "The PDF output (%s) was created, but the "
- "PostScript file which was used to make it (%s) could not be"
- "removed", p->cp.output, epsfilename);
- free(epsfilename);
- }
-}
diff --git a/bin/convertt/eps.h b/bin/convertt/eps.h
deleted file mode 100644
index b9eba01..0000000
--- a/bin/convertt/eps.h
+++ /dev/null
@@ -1,41 +0,0 @@
-/*********************************************************************
-ConvertType - Convert between various types of files.
-ConvertType is part of GNU Astronomy Utilities (Gnuastro) package.
-
-Original author:
- Mohammad Akhlaghi <mohammad@akhlaghi.org>
-Contributing author(s):
-Copyright (C) 2015-2018, Free Software Foundation, Inc.
-
-Gnuastro is free software: you can redistribute it and/or modify it
-under the terms of the GNU General Public License as published by the
-Free Software Foundation, either version 3 of the License, or (at your
-option) any later version.
-
-Gnuastro is distributed in the hope that it will be useful, but
-WITHOUT ANY WARRANTY; without even the implied warranty of
-MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the GNU
-General Public License for more details.
-
-You should have received a copy of the GNU General Public License
-along with Gnuastro. If not, see <http://www.gnu.org/licenses/>.
-**********************************************************************/
-#ifndef EPS_H
-#define EPS_H
-
-int
-nameiseps(char *name);
-
-int
-nameisepssuffix(char *name);
-
-int
-nameispdf(char *name);
-
-int
-nameispdfsuffix(char *name);
-
-void
-eps_write_eps_or_pdf(struct converttparams *p);
-
-#endif
diff --git a/bin/convertt/ui.c b/bin/convertt/ui.c
index e6a7aec..0df8390 100644
--- a/bin/convertt/ui.c
+++ b/bin/convertt/ui.c
@@ -28,10 +28,14 @@ along with Gnuastro. If not, see
<http://www.gnu.org/licenses/>.
#include <stdio.h>
#include <string.h>
+#include <gnuastro/eps.h>
#include <gnuastro/txt.h>
#include <gnuastro/wcs.h>
+#include <gnuastro/pdf.h>
#include <gnuastro/list.h>
#include <gnuastro/fits.h>
+#include <gnuastro/jpeg.h>
+#include <gnuastro/tiff.h>
#include <gnuastro/table.h>
#include <gnuastro/blank.h>
#include <gnuastro/arithmetic.h>
@@ -44,8 +48,6 @@ along with Gnuastro. If not, see
<http://www.gnu.org/licenses/>.
#include "main.h"
#include "ui.h"
-#include "eps.h"
-#include "jpeg.h"
#include "authors-cite.h"
@@ -393,9 +395,9 @@ static void
ui_make_channels_ll(struct converttparams *p)
{
char *hdu=NULL;
- size_t dsize=0;
gal_data_t *data;
gal_list_str_t *name;
+ size_t dsize=0, dirnum;
/* Go through the input files and add the channel(s). */
p->numch=0;
@@ -423,7 +425,8 @@ ui_make_channels_ll(struct converttparams *p)
"for each input FITS image (in the same order)");
/* Read in the array and its WCS information. */
- data=gal_fits_img_read(name->v, hdu, p->cp.minmapsize, 0, 0);
+ data=gal_fits_img_read(name->v, hdu, p->cp.minmapsize);
+ data->wcs=gal_wcs_read(name->v, hdu, 0, 0, &data->nwcs);
gal_list_data_add(&p->chll, data);
/* A FITS file only has one channel. */
@@ -431,19 +434,31 @@ ui_make_channels_ll(struct converttparams *p)
}
+ /* TIFF: */
+ else if( gal_tiff_name_is_tiff(name->v) )
+ {
+ /* Get the directory value for this channel. */
+ if(p->hdus)
+ {
+ hdu=gal_list_str_pop(&p->hdus);
+ dirnum=gal_tiff_dir_string_read(hdu);
+ }
+ else
+ dirnum=0;
+
+ /* Read the TIFF image into memory. */
+ data=gal_tiff_read(name->v, dirnum, p->cp.minmapsize);
+ p->numch += gal_list_data_number(data);
+ gal_list_data_add(&p->chll, data);
+ }
+
/* JPEG: */
- else if ( nameisjpeg(name->v) )
+ else if ( gal_jpeg_name_is_jpeg(name->v) )
{
-#ifndef HAVE_LIBJPEG
- error(EXIT_FAILURE, 0, "you are giving a JPEG input, however, "
- "when %s was configured libjpeg was not available. To read "
- "from (and write to) JPEG files, libjpeg is required. "
- "Please install it and configure, make and install %s "
- "again", PACKAGE_STRING, PACKAGE_STRING);
-#else
- p->numch += jpeg_read_to_ll(name->v, &p->chll, p->cp.minmapsize);
-#endif
+ data=gal_jpeg_read(name->v, p->cp.minmapsize);
+ p->numch += gal_list_data_number(data);
+ gal_list_data_add(&p->chll, data);
}
@@ -460,18 +475,18 @@ ui_make_channels_ll(struct converttparams *p)
/* EPS: */
- else if ( nameiseps(name->v) )
+ else if ( gal_eps_name_is_eps(name->v) )
error(EXIT_FAILURE, 0, "EPS files cannot be used as input. Since "
- "EPS files are not raster graphics, they are only used as "
- "output");
+ "EPS files are not raster graphics. EPS is only an output "
+ "format");
/* PDF: */
- else if ( nameispdf(name->v) )
+ else if ( gal_pdf_name_is_pdf(name->v) )
error(EXIT_FAILURE, 0, "PDF files cannot be used as input. Since "
- "PDF files are not raster graphics, they are only used as "
- "output");
+ "PDF files are not raster graphics. PDF is only an output "
+ "format");
/* Text: */
@@ -636,49 +651,55 @@ ui_set_output(struct converttparams *p)
{
struct gal_options_common_params *cp=&p->cp;
- /* Determine the type */
+ /* FITS */
if(gal_fits_name_is_fits(cp->output))
{
p->outformat=OUT_FORMAT_FITS;
if( gal_fits_suffix_is_fits(cp->output) )
ui_add_dot_use_automatic_output(p);
}
- else if(nameisjpeg(cp->output))
+
+ /* JPEG */
+ else if(gal_jpeg_name_is_jpeg(cp->output))
{
-#ifndef HAVE_LIBJPEG
- error(EXIT_FAILURE, 0, "you have asked for a JPEG output, "
- "however, when %s was configured libjpeg was not "
- "available. To write to JPEG files, libjpeg is required. "
- "Please install it and configure, make and install %s "
- "again", PACKAGE_STRING, PACKAGE_STRING);
-#else
/* Small sanity checks. */
if(p->quality == GAL_BLANK_UINT8)
- error(EXIT_FAILURE, 0, "the `--quality' (`-u') option is necessary for
"
- "jpeg outputs, but it has not been given");
+ error(EXIT_FAILURE, 0, "the `--quality' (`-u') option is necessary "
+ "for jpeg outputs, but it has not been given");
if(p->quality > 100)
error(EXIT_FAILURE, 0, "`%u' is larger than 100. The value to the "
- "`--quality' (`-u') option must be between 1 and 100
(inclusive)",
- p->quality);
+ "`--quality' (`-u') option must be between 1 and 100 "
+ "(inclusive)", p->quality);
/* Preparations. */
p->outformat=OUT_FORMAT_JPEG;
- if( nameisjpegsuffix(cp->output) )
+ if( gal_jpeg_suffix_is_jpeg(cp->output) )
ui_add_dot_use_automatic_output(p);
-#endif
}
- else if(nameiseps(cp->output))
+
+ /* TIFF */
+ else if( gal_tiff_name_is_tiff(cp->output) )
+ error(EXIT_FAILURE, 0, "writing TIFF files is not yet supported, "
+ "please get in touch with us at %s so we implement it",
+ PACKAGE_BUGREPORT);
+
+ /* EPS */
+ else if(gal_eps_name_is_eps(cp->output))
{
p->outformat=OUT_FORMAT_EPS;
- if( nameisepssuffix(cp->output) )
+ if( gal_eps_suffix_is_eps(cp->output) )
ui_add_dot_use_automatic_output(p);
}
- else if(nameispdf(cp->output))
+
+ /* PDF */
+ else if(gal_pdf_name_is_pdf(cp->output))
{
p->outformat=OUT_FORMAT_PDF;
- if( nameispdfsuffix(cp->output) )
+ if( gal_pdf_suffix_is_pdf(cp->output) )
ui_add_dot_use_automatic_output(p);
}
+
+ /* Default: plain text. */
else
{
p->outformat=OUT_FORMAT_TXT;
@@ -715,7 +736,7 @@ ui_set_output(struct converttparams *p)
}
}
- /* Check if the output already exists. */
+ /* Check if the output already exists and remove it if allowed. */
gal_checkset_writable_remove(cp->output, 0, cp->dontdelete);
}
diff --git a/bin/convolve/ui.c b/bin/convolve/ui.c
index 831a7e6..fe9cc53 100644
--- a/bin/convolve/ui.c
+++ b/bin/convolve/ui.c
@@ -33,6 +33,7 @@ along with Gnuastro. If not, see
<http://www.gnu.org/licenses/>.
#include <gnuastro/tile.h>
#include <gnuastro/blank.h>
#include <gnuastro/table.h>
+#include <gnuastro/array.h>
#include <gnuastro/threads.h>
#include <gnuastro/arithmetic.h>
#include <gnuastro/statistics.h>
@@ -294,10 +295,9 @@ ui_read_kernel(struct convolveparams *p)
float *f, *ff;
/* Read the image into file. */
- p->kernel = gal_fits_img_read_to_type(p->kernelname, p->khdu,
- GAL_TYPE_FLOAT32, p->cp.minmapsize,
- 0, 0);
- if(p->kernel->wcs) { wcsfree(p->kernel->wcs); p->kernel->wcs=NULL; }
+ p->kernel = gal_array_read_one_ch_to_type(p->kernelname, p->khdu,
+ GAL_TYPE_FLOAT32,
+ p->cp.minmapsize);
/* Convert all the NaN pixels to zero if the kernel contains blank
pixels, also update the flags so it is not checked any more. */
@@ -343,8 +343,9 @@ ui_preparations(struct convolveparams *p)
/* Read the input image as a float64 array. */
- p->input=gal_fits_img_read_to_type(p->filename, cp->hdu,
- GAL_TYPE_FLOAT32, cp->minmapsize, 0, 0);
+ p->input=gal_array_read_one_ch_to_type(p->filename, cp->hdu,
+ GAL_TYPE_FLOAT32, cp->minmapsize);
+ p->input->wcs=gal_wcs_read(p->filename, cp->hdu, 0, 0, &p->input->nwcs);
/* Domain specific checks. */
diff --git a/bin/fits/args.h b/bin/fits/args.h
index a792d6c..76baf3c 100644
--- a/bin/fits/args.h
+++ b/bin/fits/args.h
@@ -157,7 +157,7 @@ struct argp_option program_options[] =
"Add HISTORY keyword, any length is ok.",
UI_GROUP_KEYWORD,
&p->history,
- GAL_TYPE_STRING,
+ GAL_TYPE_STRLL,
GAL_OPTIONS_RANGE_ANY,
GAL_OPTIONS_NOT_MANDATORY,
GAL_OPTIONS_NOT_SET
@@ -170,7 +170,7 @@ struct argp_option program_options[] =
"Add COMMENT keyword, any length is ok.",
UI_GROUP_KEYWORD,
&p->comment,
- GAL_TYPE_STRING,
+ GAL_TYPE_STRLL,
GAL_OPTIONS_RANGE_ANY,
GAL_OPTIONS_NOT_MANDATORY,
GAL_OPTIONS_NOT_SET
diff --git a/bin/fits/fits.c b/bin/fits/fits.c
index b836dd4..895e7c0 100644
--- a/bin/fits/fits.c
+++ b/bin/fits/fits.c
@@ -64,11 +64,11 @@ fits_has_error(struct fitsparams *p, int actioncode, char
*string, int status)
if(p->quitonerror)
{
fits_report_error(stderr, status);
- error(EXIT_FAILURE, 0, "%s: not %s: %s\n", __func__, action, string);
+ error(EXIT_FAILURE, 0, "%s: %s: not %s\n", __func__, string, action);
}
else
{
- fprintf(stderr, "Not %s: %s\n", action, string);
+ fprintf(stderr, "%s: Not %s.\n", string, action);
r=EXIT_FAILURE;
}
return r;
@@ -259,6 +259,7 @@ fits_hdu_remove(struct fitsparams *p, int *r)
/* Delete the extension. */
if( fits_delete_hdu(fptr, &hdutype, &status) )
*r=fits_has_error(p, FITS_ACTION_REMOVE, hdu, status);
+ status=0;
/* Close the file. */
fits_close_file(fptr, &status);
@@ -293,11 +294,15 @@ fits_hdu_copy(struct fitsparams *p, int cut1_copy0, int
*r)
/* Copy to the extension. */
if( fits_copy_hdu(in, out, 0, &status) )
*r=fits_has_error(p, FITS_ACTION_COPY, hdu, status);
+ status=0;
/* If this is a `cut' operation, then remove the extension. */
if(cut1_copy0)
- if( fits_delete_hdu(in, &hdutype, &status) )
- *r=fits_has_error(p, FITS_ACTION_REMOVE, hdu, status);
+ {
+ if( fits_delete_hdu(in, &hdutype, &status) )
+ *r=fits_has_error(p, FITS_ACTION_REMOVE, hdu, status);
+ status=0;
+ }
/* Close the input file. */
fits_close_file(in, &status);
diff --git a/bin/fits/keywords.c b/bin/fits/keywords.c
index cc0a80a..366a5fc 100644
--- a/bin/fits/keywords.c
+++ b/bin/fits/keywords.c
@@ -109,6 +109,7 @@ keywords_rename_keys(struct fitsparams *p, fitsfile **fptr,
int *r)
/* Rename the keyword */
fits_modify_name(*fptr, from, to, &status);
if(status) *r=fits_has_error(p, FITS_ACTION_RENAME, from, status);
+ status=0;
/* Clean up the user's input string. Note that `strtok' just changes
characters within the allocated string, no extra allocation is
@@ -208,9 +209,6 @@ keywords_print_all_keys(struct fitsparams *p, fitsfile
**fptr)
int nkeys, status=0;
char *fullheader, *c, *cf;
- /* Open the FITS file. */
- keywords_open(p, fptr, READONLY);
-
/* Conver the header into a contiguous string. */
if( fits_hdr2str(*fptr, 0, NULL, 0, &fullheader, &nkeys, &status) )
gal_fits_io_error(status, NULL);
@@ -257,6 +255,15 @@ keywords_print_all_keys(struct fitsparams *p, fitsfile
**fptr)
/***********************************************************************/
/****************** Main function ********************/
/***********************************************************************/
+/* NOTE ON CALLING keywords_open FOR EACH OPERATION:
+
+ `keywords_open' is being called individually for each separate operation
+ because the necessary permissions differ: when the user only wants to
+ read keywords, they don't necessarily need write permissions. So if they
+ haven't asked for any writing/editing operation, we shouldn't open in
+ write-mode. Because the user might not have the permissions to write and
+ they might not want to write. `keywords_open' will only open the file
+ once (if the pointer is already allocated, it won't do anything). */
int
keywords(struct fitsparams *p)
{
@@ -269,68 +276,101 @@ keywords(struct fitsparams *p)
/* Delete the requested keywords. */
if(p->delete)
{
+ /* Open the FITS file. */
keywords_open(p, &fptr, READWRITE);
+
+ /* Go over all the keywords to delete. */
for(tstll=p->delete; tstll!=NULL; tstll=tstll->next)
{
fits_delete_key(fptr, tstll->v, &status);
if(status)
r=fits_has_error(p, FITS_ACTION_DELETE, tstll->v, status);
+ status=0;
}
}
/* Rename the requested keywords. */
if(p->rename)
- keywords_rename_keys(p, &fptr, &r);
+ {
+ keywords_open(p, &fptr, READWRITE);
+ keywords_rename_keys(p, &fptr, &r);
+ }
/* Update the requested keywords. */
if(p->update)
- keywords_write_update(p, &fptr, p->update_keys, 1);
+ {
+ keywords_open(p, &fptr, READWRITE);
+ keywords_write_update(p, &fptr, p->update_keys, 1);
+ }
/* Write the requested keywords. */
if(p->write)
- keywords_write_update(p, &fptr, p->write_keys, 2);
+ {
+ keywords_open(p, &fptr, READWRITE);
+ keywords_write_update(p, &fptr, p->write_keys, 2);
+ }
/* Put in any full line of keywords as-is. */
if(p->asis)
- for(tstll=p->asis; tstll!=NULL; tstll=tstll->next)
- {
- fits_write_record(fptr, tstll->v, &status);
- if(status) r=fits_has_error(p, FITS_ACTION_WRITE, tstll->v, status);
- }
+ {
+ keywords_open(p, &fptr, READWRITE);
+ for(tstll=p->asis; tstll!=NULL; tstll=tstll->next)
+ {
+ fits_write_record(fptr, tstll->v, &status);
+ if(status) r=fits_has_error(p, FITS_ACTION_WRITE, tstll->v, status);
+ status=0;
+ }
+ }
/* Add the history keyword(s). */
if(p->history)
{
- fits_write_history(fptr, p->history, &status);
- if(status) r=fits_has_error(p, FITS_ACTION_WRITE, "HISTORY", status);
+ keywords_open(p, &fptr, READWRITE);
+ for(tstll=p->history; tstll!=NULL; tstll=tstll->next)
+ {
+ fits_write_history(fptr, tstll->v, &status);
+ if(status)
+ r=fits_has_error(p, FITS_ACTION_WRITE, "HISTORY", status);
+ status=0;
+ }
}
/* Add comment(s). */
if(p->comment)
{
- fits_write_comment(fptr, p->comment, &status);
- if(status) r=fits_has_error(p, FITS_ACTION_WRITE, "COMMENT", status);
+ keywords_open(p, &fptr, READWRITE);
+ for(tstll=p->comment; tstll!=NULL; tstll=tstll->next)
+ {
+ fits_write_comment(fptr, tstll->v, &status);
+ if(status)
+ r=fits_has_error(p, FITS_ACTION_WRITE, "COMMENT", status);
+ status=0;
+ }
}
/* Update/add the date. */
if(p->date)
{
+ keywords_open(p, &fptr, READWRITE);
fits_write_date(fptr, &status);
if(status) r=fits_has_error(p, FITS_ACTION_WRITE, "DATE", status);
+ status=0;
}
- /* If nothing was requested, then print all the keywords in this
- extension. */
+ /* Print all the keywords in the extension. */
if(p->printallkeys)
- keywords_print_all_keys(p, &fptr);
+ {
+ keywords_open(p, &fptr, READONLY);
+ keywords_print_all_keys(p, &fptr);
+ }
/* Close the FITS file */
diff --git a/bin/fits/main.h b/bin/fits/main.h
index 3117d72..78681f0 100644
--- a/bin/fits/main.h
+++ b/bin/fits/main.h
@@ -61,13 +61,13 @@ struct fitsparams
gal_list_str_t *cut; /* Copy ext. to output and remove. */
uint8_t printallkeys; /* Print all the header keywords. */
uint8_t date; /* Set DATE to current time. */
- char *comment; /* COMMENT value. */
- char *history; /* HISTORY value. */
gal_list_str_t *asis; /* Strings to write asis. */
gal_list_str_t *delete; /* Keywords to remove. */
gal_list_str_t *rename; /* Rename a keyword. */
gal_list_str_t *update; /* For keywords to update. */
gal_list_str_t *write; /* Full arg. for keywords to add. */
+ gal_list_str_t *history; /* HISTORY value. */
+ gal_list_str_t *comment; /* COMMENT value. */
uint8_t quitonerror; /* Quit if an error occurs. */
/* Internal: */
diff --git a/bin/mkcatalog/Makefile.am b/bin/mkcatalog/Makefile.am
index a929552..3576618 100644
--- a/bin/mkcatalog/Makefile.am
+++ b/bin/mkcatalog/Makefile.am
@@ -30,10 +30,10 @@ bin_PROGRAMS = astmkcatalog
astmkcatalog_LDADD = -lgnuastro
-astmkcatalog_SOURCES = main.c ui.c mkcatalog.c columns.c upperlimit.c
+astmkcatalog_SOURCES = main.c ui.c mkcatalog.c columns.c upperlimit.c parse.c
EXTRA_DIST = main.h authors-cite.h args.h ui.h mkcatalog.h columns.h \
- upperlimit.h
+ upperlimit.h parse.h
diff --git a/bin/mkcatalog/args.h b/bin/mkcatalog/args.h
index e5be022..feb5954 100644
--- a/bin/mkcatalog/args.h
+++ b/bin/mkcatalog/args.h
@@ -33,63 +33,63 @@ struct argp_option program_options[] =
{
/* Input options. */
{
- "objectsfile",
- UI_KEY_OBJECTSFILE,
+ "clumpsfile",
+ UI_KEY_CLUMPSFILE,
"STR",
0,
- "Image containing object/detection labels.",
+ "Dataset containing clump labels.",
GAL_OPTIONS_GROUP_INPUT,
- &p->objectsfile,
+ &p->clumpsfile,
GAL_TYPE_STRING,
GAL_OPTIONS_RANGE_ANY,
GAL_OPTIONS_NOT_MANDATORY,
GAL_OPTIONS_NOT_SET
},
{
- "objectshdu",
- UI_KEY_OBJECTSHDU,
+ "clumpshdu",
+ UI_KEY_CLUMPSHDU,
"STR",
0,
- "Object image extension name or number.",
+ "Clump labels extension name or number.",
GAL_OPTIONS_GROUP_INPUT,
- &p->objectshdu,
+ &p->clumpshdu,
GAL_TYPE_STRING,
GAL_OPTIONS_RANGE_ANY,
- GAL_OPTIONS_MANDATORY,
+ GAL_OPTIONS_NOT_MANDATORY,
GAL_OPTIONS_NOT_SET
},
{
- "clumpsfile",
- UI_KEY_CLUMPSFILE,
+ "valuesfile",
+ UI_KEY_VALUESFILE,
"STR",
0,
- "Image containing clump labels.",
+ "Values/brightness dataset.",
GAL_OPTIONS_GROUP_INPUT,
- &p->clumpsfile,
+ &p->valuesfile,
GAL_TYPE_STRING,
GAL_OPTIONS_RANGE_ANY,
GAL_OPTIONS_NOT_MANDATORY,
GAL_OPTIONS_NOT_SET
},
{
- "clumpshdu",
- UI_KEY_CLUMPSHDU,
+ "valueshdu",
+ UI_KEY_VALUESHDU,
"STR",
0,
- "Clump image extension name or number.",
+ "Name or number of extension containing values.",
GAL_OPTIONS_GROUP_INPUT,
- &p->clumpshdu,
+ &p->valueshdu,
GAL_TYPE_STRING,
GAL_OPTIONS_RANGE_ANY,
GAL_OPTIONS_NOT_MANDATORY,
GAL_OPTIONS_NOT_SET
},
{
- "skyfile",
- UI_KEY_SKYFILE,
- "STR",
+ "insky",
+ UI_KEY_INSKY,
+ "STR/FLT",
0,
- "Image containing sky values.",
+ "Input Sky value or dataset.",
GAL_OPTIONS_GROUP_INPUT,
&p->skyfile,
GAL_TYPE_STRING,
@@ -107,15 +107,28 @@ struct argp_option program_options[] =
&p->skyhdu,
GAL_TYPE_STRING,
GAL_OPTIONS_RANGE_ANY,
- GAL_OPTIONS_MANDATORY,
+ GAL_OPTIONS_NOT_MANDATORY,
GAL_OPTIONS_NOT_SET
},
{
- "stdfile",
- UI_KEY_STDFILE,
- "STR",
+ "subtractsky",
+ UI_KEY_SUBTRACTSKY,
+ 0,
0,
- "Image containing sky STD values.",
+ "Subtract the Sky dataset from the values.",
+ GAL_OPTIONS_GROUP_INPUT,
+ &p->subtractsky,
+ GAL_OPTIONS_NO_ARG_TYPE,
+ GAL_OPTIONS_RANGE_0_OR_1,
+ GAL_OPTIONS_NOT_MANDATORY,
+ GAL_OPTIONS_NOT_SET
+ },
+ {
+ "instd",
+ UI_KEY_INSTD,
+ "STR/FLT",
+ 0,
+ "Sky standard deviation value or dataset.",
GAL_OPTIONS_GROUP_INPUT,
&p->stdfile,
GAL_TYPE_STRING,
@@ -128,12 +141,25 @@ struct argp_option program_options[] =
UI_KEY_STDHDU,
"STR",
0,
- "Sky image extension name or number.",
+ "Sky STD extension name or number.",
GAL_OPTIONS_GROUP_INPUT,
&p->stdhdu,
GAL_TYPE_STRING,
GAL_OPTIONS_RANGE_ANY,
- GAL_OPTIONS_MANDATORY,
+ GAL_OPTIONS_NOT_MANDATORY,
+ GAL_OPTIONS_NOT_SET
+ },
+ {
+ "variance",
+ UI_KEY_VARIANCE,
+ 0,
+ 0,
+ "STD input dataset is actually variance.",
+ GAL_OPTIONS_GROUP_INPUT,
+ &p->variance,
+ GAL_OPTIONS_NO_ARG_TYPE,
+ GAL_OPTIONS_RANGE_0_OR_1,
+ GAL_OPTIONS_NOT_MANDATORY,
GAL_OPTIONS_NOT_SET
},
{
@@ -149,36 +175,36 @@ struct argp_option program_options[] =
GAL_OPTIONS_NOT_MANDATORY,
GAL_OPTIONS_NOT_SET
},
+
+
+
+ /* Output. */
{
- "skysubtracted",
- UI_KEY_SKYSUBTRACTED,
+ "clumpscat",
+ UI_KEY_CLUMPSCAT,
0,
0,
- "Input is already sky subtracted (for S/N).",
- GAL_OPTIONS_GROUP_INPUT,
- &p->skysubtracted,
+ "Make a clumps catalog also.",
+ GAL_OPTIONS_GROUP_OUTPUT,
+ &p->clumpscat,
GAL_OPTIONS_NO_ARG_TYPE,
GAL_OPTIONS_RANGE_0_OR_1,
GAL_OPTIONS_NOT_MANDATORY,
GAL_OPTIONS_NOT_SET
},
{
- "threshold",
- UI_KEY_THRESHOLD,
- "FLT",
+ "noclumpsort",
+ UI_KEY_NOCLUMPSORT,
0,
- "Use pixels more than this multiple of STD.",
- GAL_OPTIONS_GROUP_INPUT,
- &p->threshold,
- GAL_TYPE_FLOAT32,
- GAL_OPTIONS_RANGE_ANY,
+ 0,
+ "Don't sort the clumps catalog by ID.",
+ GAL_OPTIONS_GROUP_OUTPUT,
+ &p->noclumpsort,
+ GAL_OPTIONS_NO_ARG_TYPE,
+ GAL_OPTIONS_RANGE_0_OR_1,
GAL_OPTIONS_NOT_MANDATORY,
GAL_OPTIONS_NOT_SET
},
-
-
-
- /* Output. */
{
"sfmagnsigma",
UI_KEY_SFMAGNSIGMA,
@@ -307,6 +333,20 @@ struct argp_option program_options[] =
GAL_OPTIONS_MANDATORY,
GAL_OPTIONS_NOT_SET
},
+ {
+ "checkupperlimit",
+ UI_KEY_CHECKUPPERLIMIT,
+ "INT[,INT]",
+ 0,
+ "Check random distribution for one label.",
+ UI_GROUP_UPPERLIMIT,
+ &p->checkupperlimit,
+ GAL_TYPE_STRING,
+ GAL_OPTIONS_RANGE_GT_0,
+ GAL_OPTIONS_NOT_MANDATORY,
+ GAL_OPTIONS_NOT_SET,
+ ui_check_upperlimit
+ },
@@ -787,6 +827,20 @@ struct argp_option program_options[] =
ui_column_codes_ll
},
{
+ "brightnesserr",
+ UI_KEY_BRIGHTNESSERR,
+ 0,
+ 0,
+ "Error (1-sigma) in measuring brightness.",
+ UI_GROUP_COLUMNS_BRIGHTNESS,
+ 0,
+ GAL_TYPE_INVALID,
+ GAL_OPTIONS_RANGE_ANY,
+ GAL_OPTIONS_NOT_MANDATORY,
+ GAL_OPTIONS_NOT_SET,
+ ui_column_codes_ll
+ },
+ {
"clumpbrightness",
UI_KEY_CLUMPSBRIGHTNESS,
0,
@@ -931,7 +985,7 @@ struct argp_option program_options[] =
UI_KEY_UPPERLIMITSIGMA,
0,
0,
- "Place in upperlimit distribution (sigma multiple).",
+ "Place in random distribution (sigma multiple).",
UI_GROUP_COLUMNS_BRIGHTNESS,
0,
GAL_TYPE_INVALID,
@@ -955,6 +1009,20 @@ struct argp_option program_options[] =
ui_column_codes_ll
},
{
+ "upperlimitskew",
+ UI_KEY_UPPERLIMITSKEW,
+ 0,
+ 0,
+ "(Mean-Median)/STD of random distribution.",
+ UI_GROUP_COLUMNS_BRIGHTNESS,
+ 0,
+ GAL_TYPE_INVALID,
+ GAL_OPTIONS_RANGE_ANY,
+ GAL_OPTIONS_NOT_MANDATORY,
+ GAL_OPTIONS_NOT_SET,
+ ui_column_codes_ll
+ },
+ {
"riverave",
UI_KEY_RIVERAVE,
0,
@@ -1015,7 +1083,7 @@ struct argp_option program_options[] =
UI_KEY_STD,
0,
0,
- "Average Sky standard deviation.",
+ "Average Sky standard deviation under label.",
UI_GROUP_COLUMNS_BRIGHTNESS,
0,
GAL_TYPE_INVALID,
@@ -1053,7 +1121,7 @@ struct argp_option program_options[] =
UI_KEY_AREA,
0,
0,
- "Number of pixels in clump or object.",
+ "Number of non-blank valued pixels.",
UI_GROUP_COLUMNS_MORPHOLOGY,
0,
GAL_TYPE_INVALID,
@@ -1067,7 +1135,7 @@ struct argp_option program_options[] =
UI_KEY_CLUMPSAREA,
0,
0,
- "Sum of all clump areas in an object.",
+ "Non-blank area covered by clumps.",
UI_GROUP_COLUMNS_MORPHOLOGY,
0,
GAL_TYPE_INVALID,
@@ -1081,7 +1149,21 @@ struct argp_option program_options[] =
UI_KEY_WEIGHTAREA,
0,
0,
- "Area used for flux weighted positions.",
+ "Area used for value weighted positions.",
+ UI_GROUP_COLUMNS_MORPHOLOGY,
+ 0,
+ GAL_TYPE_INVALID,
+ GAL_OPTIONS_RANGE_ANY,
+ GAL_OPTIONS_NOT_MANDATORY,
+ GAL_OPTIONS_NOT_SET,
+ ui_column_codes_ll
+ },
+ {
+ "geoarea",
+ UI_KEY_GEOAREA,
+ 0,
+ 0,
+ "Area labled region (irrespective of value).",
UI_GROUP_COLUMNS_MORPHOLOGY,
0,
GAL_TYPE_INVALID,
diff --git a/bin/mkcatalog/astmkcatalog.conf b/bin/mkcatalog/astmkcatalog.conf
index b2d111f..79a9ce2 100644
--- a/bin/mkcatalog/astmkcatalog.conf
+++ b/bin/mkcatalog/astmkcatalog.conf
@@ -18,13 +18,12 @@
# warranty.
# Input:
- hdu 1
- objectshdu 2
- clumpshdu 3
+ hdu OBJECTS
+ valueshdu 1
+ clumpshdu CLUMPS
skyhdu SKY
stdhdu SKY_STD
zeropoint 0.0
- skysubtracted 0
# Output:
sfmagnsigma 1
diff --git a/bin/mkcatalog/columns.c b/bin/mkcatalog/columns.c
index 6ed9670..b09d64e 100644
--- a/bin/mkcatalog/columns.c
+++ b/bin/mkcatalog/columns.c
@@ -59,14 +59,14 @@ columns_alloc_radec(struct mkcatalogparams *p)
/* For objects. */
if(p->wcs_vo==NULL)
- for(i=0;i<p->input->ndim;++i)
+ for(i=0;i<p->objects->ndim;++i)
gal_list_data_add_alloc(&p->wcs_vo, NULL, GAL_TYPE_FLOAT64, 1,
&p->numobjects, NULL, 0, p->cp.minmapsize,
NULL, NULL, NULL);
/* For clumps */
if(p->clumps && p->wcs_vc==NULL)
- for(i=0;i<p->input->ndim;++i)
+ for(i=0;i<p->objects->ndim;++i)
gal_list_data_add_alloc(&p->wcs_vc, NULL, GAL_TYPE_FLOAT64, 1,
&p->numclumps, NULL, 0, p->cp.minmapsize,
NULL, NULL, NULL);
@@ -84,14 +84,14 @@ columns_alloc_georadec(struct mkcatalogparams *p)
/* For objects. */
if(p->wcs_go==NULL)
- for(i=0;i<p->input->ndim;++i)
+ for(i=0;i<p->objects->ndim;++i)
gal_list_data_add_alloc(&p->wcs_go, NULL, GAL_TYPE_FLOAT64, 1,
&p->numobjects, NULL, 0, p->cp.minmapsize,
NULL, NULL, NULL);
/* For clumps */
if(p->clumps && p->wcs_gc==NULL)
- for(i=0;i<p->input->ndim;++i)
+ for(i=0;i<p->objects->ndim;++i)
gal_list_data_add_alloc(&p->wcs_gc, NULL, GAL_TYPE_FLOAT64, 1,
&p->numclumps, NULL, 0, p->cp.minmapsize,
NULL, NULL, NULL);
@@ -108,7 +108,7 @@ columns_alloc_clumpsradec(struct mkcatalogparams *p)
size_t i;
if(p->wcs_vcc==NULL)
- for(i=0;i<p->input->ndim;++i)
+ for(i=0;i<p->objects->ndim;++i)
gal_list_data_add_alloc(&p->wcs_vcc, NULL, GAL_TYPE_FLOAT64, 1,
&p->numobjects, NULL, 0, p->cp.minmapsize,
NULL, NULL, NULL);
@@ -125,7 +125,7 @@ columns_alloc_clumpsgeoradec(struct mkcatalogparams *p)
size_t i;
if(p->wcs_gcc==NULL)
- for(i=0;i<p->input->ndim;++i)
+ for(i=0;i<p->objects->ndim;++i)
gal_list_data_add_alloc(&p->wcs_gcc, NULL, GAL_TYPE_FLOAT64, 1,
&p->numobjects, NULL, 0, p->cp.minmapsize,
NULL, NULL, NULL);
@@ -139,10 +139,10 @@ columns_alloc_clumpsgeoradec(struct mkcatalogparams *p)
#define SET_WCS_PREPARE(ARR, LIST, ARRNAME) { \
d=0; \
errno=0; \
- (ARR)=malloc(p->input->ndim * sizeof (ARR) ); \
+ (ARR)=malloc(p->objects->ndim * sizeof (ARR) ); \
if( (ARR)==NULL ) \
error(EXIT_FAILURE, 0, "%s: %zu bytes for %s", __func__, \
- p->input->ndim * sizeof (ARR), ARRNAME); \
+ p->objects->ndim * sizeof (ARR), ARRNAME); \
for(tmp=(LIST);tmp!=NULL;tmp=tmp->next) (ARR)[d++]=tmp->array; \
}
@@ -210,13 +210,13 @@ columns_wcs_preparation(struct mkcatalogparams *p)
case UI_KEY_CLUMPSW2:
case UI_KEY_CLUMPSGEOW1:
case UI_KEY_CLUMPSGEOW2:
- if(p->input->wcs)
+ if(p->objects->wcs)
continue_wcs_check=0;
else
- error(EXIT_FAILURE, 0, "input doesn't have WCS meta-data for "
- "defining world coordinates (like RA and Dec). Atleast "
- "one of the requested columns requires this "
- "information");
+ error(EXIT_FAILURE, 0, "%s (hdu: %s): no WCS meta-data "
+ "found by WCSLIB. Atleast one of the requested columns "
+ "requires world coordinate system meta-data",
+ p->objectsfile, p->cp.hdu);
break;
}
}
@@ -231,7 +231,7 @@ columns_wcs_preparation(struct mkcatalogparams *p)
case UI_KEY_RA:
case UI_KEY_DEC:
/* Check all the CTYPES. */
- for(i=0;i<p->input->ndim;++i)
+ for(i=0;i<p->objects->ndim;++i)
if( !strcmp(p->ctype[i], colcode->v==UI_KEY_RA ? "RA" : "DEC") )
{
colcode->v = i==0 ? UI_KEY_W1 : UI_KEY_W2;
@@ -239,9 +239,9 @@ columns_wcs_preparation(struct mkcatalogparams *p)
}
/* Make sure it actually existed. */
- if(i==p->input->ndim)
+ if(i==p->objects->ndim)
error(EXIT_FAILURE, 0, "%s (hdu: %s): %s not present in any of "
- "the WCS axis types (CTYPE)", p->inputname, p->cp.hdu,
+ "the WCS axis types (CTYPE)", p->objectsfile, p->cp.hdu,
colcode->v==UI_KEY_RA ? "RA" : "DEC");
break;
}
@@ -264,7 +264,7 @@ columns_sanity_check(struct mkcatalogparams *p)
columns_wcs_preparation(p);
/* Check for dimension-specific columns. */
- switch(p->input->ndim)
+ switch(p->objects->ndim)
{
case 2:
for(colcode=p->columnids; colcode!=NULL; colcode=colcode->next)
@@ -280,7 +280,7 @@ columns_sanity_check(struct mkcatalogparams *p)
case UI_KEY_CLUMPSGEOW3:
error(EXIT_FAILURE, 0, "%s (hdu %s) is a 2D dataset, so columns "
"relating to a third dimension cannot be requested",
- p->inputname, p->cp.hdu);
+ p->objectsfile, p->cp.hdu);
}
break;
@@ -307,7 +307,7 @@ columns_sanity_check(struct mkcatalogparams *p)
error(EXIT_FAILURE, 0, "%s: a bug! Please contact us at %s to "
"fix the problem. MakeCatalog should not reach this point "
"when the input has %zu dimensions", __func__,
- PACKAGE_BUGREPORT, p->input->ndim);
+ PACKAGE_BUGREPORT, p->objects->ndim);
}
}
@@ -321,6 +321,7 @@ void
columns_define_alloc(struct mkcatalogparams *p)
{
gal_list_i32_t *colcode;
+ size_t ndim=p->objects->ndim;
gal_list_str_t *strtmp, *noclumpimg=NULL;
int disp_fmt=0, disp_width=0, disp_precision=0;
char *name=NULL, *unit=NULL, *ocomment=NULL, *ccomment=NULL;
@@ -353,7 +354,7 @@ columns_define_alloc(struct mkcatalogparams *p)
unit = "counter";
ocomment = "Object identifier.";
ccomment = NULL;
- otype = GAL_TYPE_INT32; /* Same type as clumps image. */
+ otype = GAL_TYPE_INT32;
ctype = GAL_TYPE_INVALID;
disp_fmt = 0;
disp_width = 6;
@@ -403,42 +404,53 @@ columns_define_alloc(struct mkcatalogparams *p)
case UI_KEY_AREA:
name = "AREA";
unit = "counter";
- ocomment = "Number of pixels covered.";
+ ocomment = "Number of non-blank pixels.";
ccomment = ocomment;
otype = GAL_TYPE_INT32;
ctype = GAL_TYPE_INT32;
disp_fmt = 0;
- disp_width = 5;
+ disp_width = 6;
disp_precision = 0;
- oiflag[ OCOL_NUMALL ] = 1;
- ciflag[ CCOL_NUMALL ] = 1;
+ oiflag[ OCOL_NUM ] = ciflag[ CCOL_NUM ] = 1;
break;
case UI_KEY_CLUMPSAREA:
- name = "CLUMPS_AREA";
+ name = "AREA_CLUMPS";
unit = "counter";
ocomment = "Total number of clump pixels in object.";
ccomment = NULL;
otype = GAL_TYPE_INT32;
ctype = GAL_TYPE_INVALID;
disp_fmt = 0;
- disp_width = 5;
+ disp_width = 6;
disp_precision = 0;
oiflag[ OCOL_C_NUM ] = 1;
break;
case UI_KEY_WEIGHTAREA:
- name = "WEIGHT_AREA";
+ name = "AREA_WEIGHT";
unit = "counter";
ocomment = "Area used for flux-weighted positions.";
ccomment = ocomment;
otype = GAL_TYPE_INT32;
ctype = GAL_TYPE_INT32;
disp_fmt = 0;
- disp_width = 5;
+ disp_width = 6;
+ disp_precision = 0;
+ oiflag[ OCOL_NUMWHT ] = ciflag[ CCOL_NUMWHT ] = 1;
+ break;
+
+ case UI_KEY_GEOAREA:
+ name = "AREA_FULL";
+ unit = "counter";
+ ocomment = "Full area of label (irrespective of values).";
+ ccomment = ocomment;
+ otype = GAL_TYPE_INT32;
+ ctype = GAL_TYPE_INT32;
+ disp_fmt = 0;
+ disp_width = 6;
disp_precision = 0;
- oiflag[ OCOL_NUMWHT ] = 1;
- ciflag[ CCOL_NUMWHT ] = 1;
+ oiflag[ OCOL_NUMALL ] = ciflag[ CCOL_NUMALL ] = 1;
break;
case UI_KEY_X:
@@ -451,8 +463,10 @@ columns_define_alloc(struct mkcatalogparams *p)
disp_fmt = GAL_TABLE_DISPLAY_FMT_FLOAT;
disp_width = 10;
disp_precision = 3;
- oiflag[ OCOL_VX ] = 1;
- ciflag[ CCOL_VX ] = 1;
+ oiflag[ OCOL_VX ] = ciflag[ CCOL_VX ] = 1;
+ oiflag[ OCOL_GX ] = ciflag[ CCOL_GX ] = 1;
+ oiflag[ OCOL_SUMWHT ] = ciflag[ CCOL_SUMWHT ] = 1;
+ oiflag[ OCOL_NUMWHT ] = ciflag[ CCOL_NUMWHT ] = 1;
break;
case UI_KEY_Y:
@@ -465,8 +479,10 @@ columns_define_alloc(struct mkcatalogparams *p)
disp_fmt = GAL_TABLE_DISPLAY_FMT_FLOAT;
disp_width = 10;
disp_precision = 3;
- oiflag[ OCOL_VY ] = 1;
- ciflag[ CCOL_VY ] = 1;
+ oiflag[ OCOL_VY ] = ciflag[ CCOL_VY ] = 1;
+ oiflag[ OCOL_GY ] = ciflag[ CCOL_GY ] = 1;
+ oiflag[ OCOL_SUMWHT ] = ciflag[ CCOL_SUMWHT ] = 1;
+ oiflag[ OCOL_NUMWHT ] = ciflag[ CCOL_NUMWHT ] = 1;
break;
case UI_KEY_Z:
@@ -493,8 +509,8 @@ columns_define_alloc(struct mkcatalogparams *p)
disp_fmt = GAL_TABLE_DISPLAY_FMT_FLOAT;
disp_width = 10;
disp_precision = 3;
- oiflag[ OCOL_GX ] = 1;
- ciflag[ CCOL_GX ] = 1;
+ oiflag[ OCOL_GX ] = ciflag[ CCOL_GX ] = 1;
+ oiflag[ OCOL_NUMALL ] = ciflag[ CCOL_NUMALL ] = 1;
break;
case UI_KEY_GEOY:
@@ -507,8 +523,8 @@ columns_define_alloc(struct mkcatalogparams *p)
disp_fmt = GAL_TABLE_DISPLAY_FMT_FLOAT;
disp_width = 10;
disp_precision = 3;
- oiflag[ OCOL_GY ] = 1;
- ciflag[ CCOL_GY ] = 1;
+ oiflag[ OCOL_GY ] = ciflag[ CCOL_GY ] = 1;
+ oiflag[ OCOL_NUMALL ] = ciflag[ CCOL_NUMALL ] = 1;
break;
case UI_KEY_GEOZ:
@@ -535,7 +551,10 @@ columns_define_alloc(struct mkcatalogparams *p)
disp_fmt = GAL_TABLE_DISPLAY_FMT_FLOAT;
disp_width = 10;
disp_precision = 3;
- oiflag[ OCOL_C_VX ] = 1;
+ oiflag[ OCOL_C_VX ] = 1;
+ oiflag[ OCOL_C_GX ] = 1;
+ oiflag[ OCOL_C_SUMWHT ] = 1;
+ oiflag[ OCOL_C_NUMWHT ] = 1;
break;
case UI_KEY_CLUMPSY:
@@ -548,7 +567,10 @@ columns_define_alloc(struct mkcatalogparams *p)
disp_fmt = GAL_TABLE_DISPLAY_FMT_FLOAT;
disp_width = 10;
disp_precision = 3;
- oiflag[ OCOL_C_VY ] = 1;
+ oiflag[ OCOL_C_VY ] = 1;
+ oiflag[ OCOL_C_GY ] = 1;
+ oiflag[ OCOL_C_SUMWHT ] = 1;
+ oiflag[ OCOL_C_NUMWHT ] = 1;
break;
case UI_KEY_CLUMPSZ:
@@ -574,7 +596,8 @@ columns_define_alloc(struct mkcatalogparams *p)
disp_fmt = GAL_TABLE_DISPLAY_FMT_FLOAT;
disp_width = 10;
disp_precision = 3;
- oiflag[ OCOL_C_GX ] = 1;
+ oiflag[ OCOL_C_GX ] = 1;
+ oiflag[ OCOL_C_NUMALL ] = 1;
break;
case UI_KEY_CLUMPSGEOY:
@@ -587,7 +610,8 @@ columns_define_alloc(struct mkcatalogparams *p)
disp_fmt = GAL_TABLE_DISPLAY_FMT_FLOAT;
disp_width = 10;
disp_precision = 3;
- oiflag[ OCOL_C_GY ] = 1;
+ oiflag[ OCOL_C_GY ] = 1;
+ oiflag[ OCOL_C_NUMALL ] = 1;
break;
case UI_KEY_CLUMPSGEOZ:
@@ -605,7 +629,7 @@ columns_define_alloc(struct mkcatalogparams *p)
case UI_KEY_W1:
name = p->ctype[0];
- unit = p->input->wcs->cunit[0];
+ unit = p->objects->wcs->cunit[0];
ocomment = "Flux weighted center (WCS axis 1).";
ccomment = ocomment;
otype = GAL_TYPE_FLOAT64;
@@ -613,18 +637,23 @@ columns_define_alloc(struct mkcatalogparams *p)
disp_fmt = GAL_TABLE_DISPLAY_FMT_FLOAT;
disp_width = 13;
disp_precision = 7;
- oiflag[ OCOL_VX ] = 1;
- oiflag[ OCOL_VY ] = 1;
- oiflag[ OCOL_VZ ] = 1;
- oiflag[ CCOL_VX ] = 1;
- oiflag[ CCOL_VY ] = 1;
- oiflag[ CCOL_VZ ] = 1;
columns_alloc_radec(p);
+ oiflag[ OCOL_VX ] = ciflag[ CCOL_VX ] = 1;
+ oiflag[ OCOL_VY ] = ciflag[ CCOL_VY ] = 1;
+ oiflag[ OCOL_GX ] = ciflag[ CCOL_GX ] = 1;
+ oiflag[ OCOL_GY ] = ciflag[ CCOL_GY ] = 1;
+ oiflag[ OCOL_SUMWHT ] = ciflag[ CCOL_SUMWHT ] = 1;
+ oiflag[ OCOL_NUMALL ] = ciflag[ CCOL_NUMALL ] = 1;
+ if(ndim==3)
+ {
+ oiflag[ OCOL_VZ ] = ciflag[ CCOL_VZ ] = 1;
+ oiflag[ OCOL_GZ ] = ciflag[ CCOL_GZ ] = 1;
+ }
break;
case UI_KEY_W2:
name = p->ctype[1];
- unit = p->input->wcs->cunit[1];
+ unit = p->objects->wcs->cunit[1];
ocomment = "Flux weighted center (WCS axis 2).";
ccomment = ocomment;
otype = GAL_TYPE_FLOAT64;
@@ -632,37 +661,44 @@ columns_define_alloc(struct mkcatalogparams *p)
disp_fmt = GAL_TABLE_DISPLAY_FMT_FLOAT;
disp_width = 13;
disp_precision = 7;
- oiflag[ OCOL_VX ] = 1;
- oiflag[ OCOL_VY ] = 1;
- oiflag[ OCOL_VZ ] = 1;
- oiflag[ CCOL_VX ] = 1;
- oiflag[ CCOL_VY ] = 1;
- oiflag[ CCOL_VZ ] = 1;
columns_alloc_radec(p);
+ oiflag[ OCOL_VX ] = ciflag[ CCOL_VX ] = 1;
+ oiflag[ OCOL_VY ] = ciflag[ CCOL_VY ] = 1;
+ oiflag[ OCOL_GX ] = ciflag[ CCOL_GX ] = 1;
+ oiflag[ OCOL_GY ] = ciflag[ CCOL_GY ] = 1;
+ oiflag[ OCOL_SUMWHT ] = ciflag[ CCOL_SUMWHT ] = 1;
+ oiflag[ OCOL_NUMALL ] = ciflag[ CCOL_NUMALL ] = 1;
+ if(ndim==3)
+ {
+ oiflag[ OCOL_VZ ] = ciflag[ CCOL_VZ ] = 1;
+ oiflag[ OCOL_GZ ] = ciflag[ CCOL_GZ ] = 1;
+ }
break;
case UI_KEY_W3:
name = p->ctype[2];
- unit = p->input->wcs->cunit[2];
+ unit = p->objects->wcs->cunit[2];
ocomment = "Flux weighted center (WCS axis 3).";
ccomment = ocomment;
otype = GAL_TYPE_FLOAT64;
ctype = GAL_TYPE_FLOAT64;
- disp_fmt = GAL_TABLE_DISPLAY_FMT_GENERAL;
+ disp_fmt = GAL_TABLE_DISPLAY_FMT_FLOAT;
disp_width = 13;
disp_precision = 7;
- oiflag[ OCOL_VX ] = 1;
- oiflag[ OCOL_VY ] = 1;
- oiflag[ OCOL_VZ ] = 1;
- oiflag[ CCOL_VX ] = 1;
- oiflag[ CCOL_VY ] = 1;
- oiflag[ CCOL_VZ ] = 1;
columns_alloc_radec(p);
+ oiflag[ OCOL_VX ] = ciflag[ CCOL_VX ] = 1;
+ oiflag[ OCOL_VY ] = ciflag[ CCOL_VY ] = 1;
+ oiflag[ OCOL_VZ ] = ciflag[ CCOL_VZ ] = 1;
+ oiflag[ OCOL_GX ] = ciflag[ CCOL_GX ] = 1;
+ oiflag[ OCOL_GY ] = ciflag[ CCOL_GY ] = 1;
+ oiflag[ OCOL_GZ ] = ciflag[ CCOL_GZ ] = 1;
+ oiflag[ OCOL_SUMWHT ] = ciflag[ CCOL_SUMWHT ] = 1;
+ oiflag[ OCOL_NUMALL ] = ciflag[ CCOL_NUMALL ] = 1;
break;
case UI_KEY_GEOW1:
name = gal_checkset_malloc_cat("GEO_", p->ctype[0]);
- unit = p->input->wcs->cunit[0];
+ unit = p->objects->wcs->cunit[0];
ocomment = "Geometric center (WCS axis 1).";
ccomment = ocomment;
otype = GAL_TYPE_FLOAT64;
@@ -670,18 +706,17 @@ columns_define_alloc(struct mkcatalogparams *p)
disp_fmt = GAL_TABLE_DISPLAY_FMT_FLOAT;
disp_width = 13;
disp_precision = 7;
- oiflag[ OCOL_GX ] = 1;
- oiflag[ OCOL_GY ] = 1;
- oiflag[ OCOL_GZ ] = 1;
- ciflag[ CCOL_GX ] = 1;
- ciflag[ CCOL_GY ] = 1;
- ciflag[ CCOL_GZ ] = 1;
columns_alloc_georadec(p);
+ oiflag[ OCOL_GX ] = ciflag[ CCOL_GX ] = 1;
+ oiflag[ OCOL_GY ] = ciflag[ CCOL_GY ] = 1;
+ oiflag[ OCOL_NUMALL ] = ciflag[ CCOL_NUMALL ] = 1;
+ if(ndim==3)
+ oiflag[ OCOL_GZ ] = ciflag[ CCOL_GZ ] = 1;
break;
case UI_KEY_GEOW2:
name = gal_checkset_malloc_cat("GEO_", p->ctype[1]);
- unit = p->input->wcs->cunit[1];
+ unit = p->objects->wcs->cunit[1];
ocomment = "Geometric center (WCS axis 2).";
ccomment = ocomment;
otype = GAL_TYPE_FLOAT64;
@@ -689,37 +724,34 @@ columns_define_alloc(struct mkcatalogparams *p)
disp_fmt = GAL_TABLE_DISPLAY_FMT_FLOAT;
disp_width = 13;
disp_precision = 7;
- oiflag[ OCOL_GX ] = 1;
- oiflag[ OCOL_GY ] = 1;
- oiflag[ OCOL_GZ ] = 1;
- ciflag[ CCOL_GX ] = 1;
- ciflag[ CCOL_GY ] = 1;
- ciflag[ CCOL_GZ ] = 1;
columns_alloc_georadec(p);
+ oiflag[ OCOL_GX ] = ciflag[ CCOL_GX ] = 1;
+ oiflag[ OCOL_GY ] = ciflag[ CCOL_GY ] = 1;
+ oiflag[ OCOL_NUMALL ] = ciflag[ CCOL_NUMALL ] = 1;
+ if(ndim==3)
+ oiflag[ OCOL_GZ ] = ciflag[ CCOL_GZ ] = 1;
break;
case UI_KEY_GEOW3:
name = gal_checkset_malloc_cat("GEO_", p->ctype[2]);
- unit = p->input->wcs->cunit[2];
+ unit = p->objects->wcs->cunit[2];
ocomment = "Geometric center (WCS axis 3).";
ccomment = ocomment;
otype = GAL_TYPE_FLOAT64;
ctype = GAL_TYPE_FLOAT64;
- disp_fmt = GAL_TABLE_DISPLAY_FMT_GENERAL;
+ disp_fmt = GAL_TABLE_DISPLAY_FMT_FLOAT;
disp_width = 13;
disp_precision = 7;
- oiflag[ OCOL_GX ] = 1;
- oiflag[ OCOL_GY ] = 1;
- oiflag[ OCOL_GZ ] = 1;
- ciflag[ CCOL_GX ] = 1;
- ciflag[ CCOL_GY ] = 1;
- ciflag[ CCOL_GZ ] = 1;
columns_alloc_georadec(p);
+ oiflag[ OCOL_GX ] = ciflag[ CCOL_GX ] = 1;
+ oiflag[ OCOL_GY ] = ciflag[ CCOL_GY ] = 1;
+ oiflag[ OCOL_GZ ] = ciflag[ CCOL_GZ ] = 1;
+ oiflag[ OCOL_NUMALL ] = ciflag[ CCOL_NUMALL ] = 1;
break;
case UI_KEY_CLUMPSW1:
name = gal_checkset_malloc_cat("CLUMPS_", p->ctype[0]);
- unit = p->input->wcs->cunit[0];
+ unit = p->objects->wcs->cunit[0];
ocomment = "Flux.wht center of all clumps (WCS axis 1).";
ccomment = NULL;
otype = GAL_TYPE_FLOAT64;
@@ -727,15 +759,23 @@ columns_define_alloc(struct mkcatalogparams *p)
disp_fmt = GAL_TABLE_DISPLAY_FMT_FLOAT;
disp_width = 13;
disp_precision = 7;
- oiflag[ OCOL_C_VX ] = 1;
- oiflag[ OCOL_C_VY ] = 1;
- oiflag[ OCOL_C_VZ ] = 1;
columns_alloc_clumpsradec(p);
+ oiflag[ OCOL_C_VX ] = 1;
+ oiflag[ OCOL_C_VY ] = 1;
+ oiflag[ OCOL_C_GX ] = 1;
+ oiflag[ OCOL_C_GY ] = 1;
+ oiflag[ OCOL_C_SUMWHT ] = 1;
+ oiflag[ OCOL_C_NUMALL ] = 1;
+ if(ndim==3)
+ {
+ oiflag[ OCOL_C_VZ ] = 1;
+ oiflag[ OCOL_C_GZ ] = 1;
+ }
break;
case UI_KEY_CLUMPSW2:
name = gal_checkset_malloc_cat("CLUMPS_", p->ctype[1]);
- unit = p->input->wcs->cunit[1];
+ unit = p->objects->wcs->cunit[1];
ocomment = "Flux.wht center of all clumps (WCS axis 2).";
ccomment = NULL;
otype = GAL_TYPE_FLOAT64;
@@ -743,31 +783,47 @@ columns_define_alloc(struct mkcatalogparams *p)
disp_fmt = GAL_TABLE_DISPLAY_FMT_FLOAT;
disp_width = 15;
disp_precision = 7;
- oiflag[ OCOL_C_VX ] = 1;
- oiflag[ OCOL_C_VY ] = 1;
- oiflag[ OCOL_C_VZ ] = 1;
columns_alloc_clumpsradec(p);
+ oiflag[ OCOL_C_VX ] = 1;
+ oiflag[ OCOL_C_VY ] = 1;
+ oiflag[ OCOL_C_GX ] = 1;
+ oiflag[ OCOL_C_GY ] = 1;
+ oiflag[ OCOL_C_SUMWHT ] = 1;
+ oiflag[ OCOL_C_NUMALL ] = 1;
+ if(ndim==3)
+ {
+ oiflag[ OCOL_C_VZ ] = 1;
+ oiflag[ OCOL_C_GZ ] = 1;
+ }
break;
case UI_KEY_CLUMPSW3:
name = gal_checkset_malloc_cat("CLUMPS_", p->ctype[2]);
- unit = p->input->wcs->cunit[2];
+ unit = p->objects->wcs->cunit[2];
ocomment = "Flux.wht center of all clumps (WCS axis 3).";
ccomment = NULL;
otype = GAL_TYPE_FLOAT64;
ctype = GAL_TYPE_INVALID;
- disp_fmt = GAL_TABLE_DISPLAY_FMT_GENERAL;
+ disp_fmt = GAL_TABLE_DISPLAY_FMT_FLOAT;
disp_width = 15;
disp_precision = 7;
- oiflag[ OCOL_C_VX ] = 1;
- oiflag[ OCOL_C_VY ] = 1;
- oiflag[ OCOL_C_VZ ] = 1;
columns_alloc_clumpsradec(p);
+ oiflag[ OCOL_C_VX ] = 1;
+ oiflag[ OCOL_C_VY ] = 1;
+ oiflag[ OCOL_C_VZ ] = 1;
+ oiflag[ OCOL_C_GX ] = 1;
+ oiflag[ OCOL_C_GY ] = 1;
+ oiflag[ OCOL_C_GZ ] = 1;
+ oiflag[ OCOL_C_SUMWHT ] = 1;
+ oiflag[ OCOL_C_NUMALL ] = 1;
+ if(ndim==3)
+ {
+ }
break;
case UI_KEY_CLUMPSGEOW1:
name = gal_checkset_malloc_cat("CLUMPS_GEO", p->ctype[0]);
- unit = p->input->wcs->cunit[0];
+ unit = p->objects->wcs->cunit[0];
ocomment = "Geometric center of all clumps (WCS axis 1).";
ccomment = NULL;
otype = GAL_TYPE_FLOAT64;
@@ -775,15 +831,17 @@ columns_define_alloc(struct mkcatalogparams *p)
disp_fmt = GAL_TABLE_DISPLAY_FMT_FLOAT;
disp_width = 13;
disp_precision = 7;
- oiflag[ OCOL_C_GX ] = 1;
- oiflag[ OCOL_C_GY ] = 1;
- oiflag[ OCOL_C_GZ ] = 1;
columns_alloc_clumpsgeoradec(p);
+ oiflag[ OCOL_C_GX ] = 1;
+ oiflag[ OCOL_C_GY ] = 1;
+ oiflag[ OCOL_C_NUMALL ] = 1;
+ if(ndim==3)
+ oiflag[ OCOL_C_GZ ] = 1;
break;
case UI_KEY_CLUMPSGEOW2:
name = gal_checkset_malloc_cat("CLUMPS_GEO", p->ctype[1]);
- unit = p->input->wcs->cunit[1];
+ unit = p->objects->wcs->cunit[1];
ocomment = "Geometric center of all clumps (WCS axis 2).";
ccomment = NULL;
otype = GAL_TYPE_FLOAT64;
@@ -791,104 +849,121 @@ columns_define_alloc(struct mkcatalogparams *p)
disp_fmt = GAL_TABLE_DISPLAY_FMT_FLOAT;
disp_width = 13;
disp_precision = 7;
- oiflag[ OCOL_C_GX ] = 1;
- oiflag[ OCOL_C_GY ] = 1;
- oiflag[ OCOL_C_GZ ] = 1;
columns_alloc_clumpsgeoradec(p);
+ oiflag[ OCOL_C_GX ] = 1;
+ oiflag[ OCOL_C_GY ] = 1;
+ oiflag[ OCOL_C_NUMALL ] = 1;
+ if(ndim==3)
+ oiflag[ OCOL_C_GZ ] = 1;
break;
case UI_KEY_CLUMPSGEOW3:
name = gal_checkset_malloc_cat("CLUMPS_GEO", p->ctype[2]);
- unit = p->input->wcs->cunit[2];
+ unit = p->objects->wcs->cunit[2];
ocomment = "Geometric center of all clumps (WCS axis 3).";
ccomment = NULL;
otype = GAL_TYPE_FLOAT64;
ctype = GAL_TYPE_INVALID;
- disp_fmt = GAL_TABLE_DISPLAY_FMT_GENERAL;
+ disp_fmt = GAL_TABLE_DISPLAY_FMT_FLOAT;
disp_width = 13;
disp_precision = 7;
- oiflag[ OCOL_C_GX ] = 1;
- oiflag[ OCOL_C_GY ] = 1;
- oiflag[ OCOL_C_GZ ] = 1;
columns_alloc_clumpsgeoradec(p);
+ oiflag[ OCOL_C_GX ] = 1;
+ oiflag[ OCOL_C_GY ] = 1;
+ oiflag[ OCOL_C_GZ ] = 1;
+ oiflag[ OCOL_C_NUMALL ] = 1;
break;
case UI_KEY_BRIGHTNESS:
name = "BRIGHTNESS";
- unit = p->input->unit ? p->input->unit : "pixelunit";
+ unit = MKCATALOG_NO_UNIT;
ocomment = "Brightness (sum of sky subtracted values).";
ccomment = "Brightness (sum of pixels subtracted by rivers).";
otype = GAL_TYPE_FLOAT32;
ctype = GAL_TYPE_FLOAT32;
disp_fmt = GAL_TABLE_DISPLAY_FMT_GENERAL;
disp_width = 10;
- disp_precision = 4;
- oiflag[ OCOL_SUM ] = 1;
- ciflag[ CCOL_SUM ] = 1;
- ciflag[ CCOL_RIV_NUM ] = 1;
- ciflag[ CCOL_RIV_SUM ] = 1;
+ disp_precision = 5;
+ oiflag[ OCOL_NUM ] = ciflag[ CCOL_NUM ] = 1;
+ oiflag[ OCOL_SUM ] = ciflag[ CCOL_SUM ] = 1;
+ ciflag[ CCOL_RIV_NUM ] = 1;
+ ciflag[ CCOL_RIV_SUM ] = 1;
+ break;
+
+ case UI_KEY_BRIGHTNESSERR:
+ name = "BRIGHTNESS_ERROR";
+ unit = MKCATALOG_NO_UNIT;
+ ocomment = "Error (1-sigma) in measuring brightness.";
+ ccomment = ocomment;
+ otype = GAL_TYPE_FLOAT32;
+ ctype = GAL_TYPE_FLOAT32;
+ disp_fmt = GAL_TABLE_DISPLAY_FMT_GENERAL;
+ disp_width = 10;
+ disp_precision = 5;
+ oiflag[ OCOL_NUM ] = ciflag[ CCOL_NUM ] = 1;
+ oiflag[ OCOL_SUM_VAR ] = ciflag[ CCOL_SUM_VAR ] = 1;
+ ciflag[ CCOL_RIV_NUM ] = 1;
+ ciflag[ CCOL_RIV_SUM_VAR ] = 1;
break;
case UI_KEY_CLUMPSBRIGHTNESS:
name = "CLUMPS_BRIGHTNESS";
- unit = p->input->unit ? p->input->unit : "pixelunit";
+ unit = MKCATALOG_NO_UNIT;
ocomment = "Brightness (sum of pixel values) in clumps.";
ccomment = NULL;
otype = GAL_TYPE_FLOAT32;
ctype = GAL_TYPE_INVALID;
disp_fmt = GAL_TABLE_DISPLAY_FMT_GENERAL;
disp_width = 10;
- disp_precision = 4;
+ disp_precision = 5;
+ oiflag[ OCOL_C_NUM ] = 1;
oiflag[ OCOL_C_SUM ] = 1;
break;
case UI_KEY_NORIVERBRIGHTNESS:
name = "NO_RIVER_BRIGHTNESS";
- unit = p->input->unit ? p->input->unit : "pixelunit";
+ unit = MKCATALOG_NO_UNIT;
ocomment = NULL;
ccomment = "Brightness (sum of sky subtracted values).";
otype = GAL_TYPE_INVALID;
ctype = GAL_TYPE_FLOAT32;
disp_fmt = GAL_TABLE_DISPLAY_FMT_GENERAL;
disp_width = 10;
- disp_precision = 4;
+ disp_precision = 5;
+ ciflag[ CCOL_NUM ] = 1;
ciflag[ CCOL_SUM ] = 1;
break;
case UI_KEY_MEAN:
name = "MEAN";
- unit = p->input->unit ? p->input->unit : "pixelunit";
+ unit = MKCATALOG_NO_UNIT;
ocomment = "Mean of sky subtracted values.";
ccomment = "Mean of pixels subtracted by rivers.";
otype = GAL_TYPE_FLOAT32;
ctype = GAL_TYPE_FLOAT32;
disp_fmt = GAL_TABLE_DISPLAY_FMT_GENERAL;
disp_width = 10;
- disp_precision = 4;
- oiflag[ OCOL_NUM ] = 1;
- oiflag[ OCOL_SUM ] = 1;
- ciflag[ CCOL_NUM ] = 1;
- ciflag[ CCOL_SUM ] = 1;
- ciflag[ CCOL_RIV_NUM ] = 1;
- ciflag[ CCOL_RIV_SUM ] = 1;
+ disp_precision = 5;
+ oiflag[ OCOL_NUM ] = ciflag[ CCOL_NUM ] = 1;
+ oiflag[ OCOL_SUM ] = ciflag[ CCOL_SUM ] = 1;
+ ciflag[ CCOL_RIV_NUM ] = 1;
+ ciflag[ CCOL_RIV_SUM ] = 1;
break;
case UI_KEY_MEDIAN:
name = "MEDIAN";
- unit = p->input->unit ? p->input->unit : "pixelunit";
+ unit = MKCATALOG_NO_UNIT;
ocomment = "Median of sky subtracted values.";
ccomment = "Median of pixels subtracted by rivers.";
otype = GAL_TYPE_FLOAT32;
ctype = GAL_TYPE_FLOAT32;
disp_fmt = GAL_TABLE_DISPLAY_FMT_GENERAL;
disp_width = 10;
- disp_precision = 4;
- oiflag[ OCOL_MEDIAN ] = 1;
- oiflag[ OCOL_NUMALL ] = 1;
- ciflag[ CCOL_MEDIAN ] = 1;
- ciflag[ CCOL_NUMALL ] = 1;
- ciflag[ CCOL_RIV_NUM ] = 1;
- ciflag[ CCOL_RIV_SUM ] = 1;
+ disp_precision = 5;
+ oiflag[ OCOL_NUM ] = ciflag[ CCOL_NUM ] = 1;
+ oiflag[ OCOL_MEDIAN ] = ciflag[ CCOL_MEDIAN ] = 1;
+ ciflag[ CCOL_RIV_NUM ] = 1;
+ ciflag[ CCOL_RIV_SUM ] = 1;
break;
case UI_KEY_MAGNITUDE:
@@ -901,13 +976,17 @@ columns_define_alloc(struct mkcatalogparams *p)
disp_fmt = GAL_TABLE_DISPLAY_FMT_FLOAT;
disp_width = 8;
disp_precision = 3;
- oiflag[ OCOL_SUM ] = 1;
- ciflag[ CCOL_SUM ] = 1;
p->hasmag = 1;
+ oiflag[ OCOL_NUM ] = ciflag[ CCOL_NUM ] = 1;
+ oiflag[ OCOL_SUM ] = ciflag[ CCOL_SUM ] = 1;
+ ciflag[ CCOL_SUM ] = 1;
+ ciflag[ CCOL_RIV_NUM ] = 1;
+ ciflag[ CCOL_RIV_SUM ] = 1;
+ ciflag[ CCOL_RIV_NUM ] = 1;
break;
case UI_KEY_MAGNITUDEERR:
- name = "MAGNITUDE_ERR";
+ name = "MAGNITUDE_ERROR";
unit = "log";
ocomment = "Error in measuring magnitude.";
ccomment = ocomment;
@@ -916,12 +995,10 @@ columns_define_alloc(struct mkcatalogparams *p)
disp_fmt = GAL_TABLE_DISPLAY_FMT_FLOAT;
disp_width = 8;
disp_precision = 3;
- oiflag[ OCOL_SUMSTD ] = 1;
- oiflag[ OCOL_NUM ] = 1;
- oiflag[ OCOL_SUM ] = 1;
- ciflag[ CCOL_SUMSTD ] = 1;
- ciflag[ CCOL_NUM ] = 1;
- ciflag[ CCOL_SUM ] = 1;
+ oiflag[ OCOL_SUM ] = ciflag[ CCOL_SUM ] = 1;
+ oiflag[ OCOL_SUM_VAR ] = ciflag[ CCOL_SUM_VAR ] = 1;
+ ciflag[ CCOL_RIV_SUM ] = 1;
+ ciflag[ CCOL_RIV_SUM_VAR ] = 1;
break;
case UI_KEY_CLUMPSMAGNITUDE:
@@ -934,21 +1011,27 @@ columns_define_alloc(struct mkcatalogparams *p)
disp_fmt = GAL_TABLE_DISPLAY_FMT_FLOAT;
disp_width = 8;
disp_precision = 3;
- oiflag[ OCOL_C_SUM ] = 1;
p->hasmag = 1;
+ oiflag[ OCOL_SUM ] = ciflag[ CCOL_SUM ] = 1;
+ oiflag[ OCOL_SUM_VAR ] = ciflag[ CCOL_SUM_VAR ] = 1;
+ ciflag[ CCOL_RIV_NUM ] = 1;
+ ciflag[ CCOL_RIV_SUM ] = 1;
+ ciflag[ CCOL_RIV_SUM_VAR ] = 1;
break;
case UI_KEY_UPPERLIMIT:
name = "UPPERLIMIT";
- unit = p->input->unit;
+ unit = MKCATALOG_NO_UNIT;
ocomment = "Upper limit value (random positionings).";
ccomment = ocomment;
otype = GAL_TYPE_FLOAT32;
ctype = GAL_TYPE_FLOAT32;
disp_fmt = GAL_TABLE_DISPLAY_FMT_GENERAL;
- disp_width = 8;
- disp_precision = 3;
- p->upperlimit = 1; /* Doesn't need per-pixel calculations. */
+ disp_width = 10;
+ disp_precision = 5;
+ p->upperlimit = 1;
+ oiflag[ OCOL_UPPERLIMIT_B ] = ciflag[ CCOL_UPPERLIMIT_B ] = 1;
+
break;
case UI_KEY_UPPERLIMITMAG:
@@ -961,34 +1044,42 @@ columns_define_alloc(struct mkcatalogparams *p)
disp_fmt = GAL_TABLE_DISPLAY_FMT_FLOAT;
disp_width = 8;
disp_precision = 3;
+ p->hasmag = 1;
p->upperlimit = 1;
- p->hasmag = 1; /* Doesn't need per-pixel calculations. */
+ oiflag[ OCOL_UPPERLIMIT_B ] = ciflag[ CCOL_UPPERLIMIT_B ] = 1;
break;
case UI_KEY_UPPERLIMITONESIGMA:
name = "UPPERLIMIT_ONE_SIGMA";
- unit = p->input->unit;
+ unit = MKCATALOG_NO_UNIT;
ocomment = "One sigma value of all random measurements.";
ccomment = ocomment;
otype = GAL_TYPE_FLOAT32;
ctype = GAL_TYPE_FLOAT32;
disp_fmt = GAL_TABLE_DISPLAY_FMT_GENERAL;
- disp_width = 8;
- disp_precision = 3;
+ disp_width = 10;
+ disp_precision = 5;
p->upperlimit = 1;
+ oiflag[ OCOL_UPPERLIMIT_S ] = ciflag[ CCOL_UPPERLIMIT_S ] = 1;
break;
case UI_KEY_UPPERLIMITSIGMA:
name = "UPPERLIMIT_SIGMA";
unit = "frac";
- ocomment = "Place in upperlimit distribution (sigma
multiple).";
+ ocomment = "Place in upperlimit distribution (sigma "
+ "multiple).";
ccomment = ocomment;
otype = GAL_TYPE_FLOAT32;
ctype = GAL_TYPE_FLOAT32;
disp_fmt = GAL_TABLE_DISPLAY_FMT_GENERAL;
- disp_width = 8;
- disp_precision = 3;
+ disp_width = 10;
+ disp_precision = 5;
p->upperlimit = 1;
+ oiflag[ OCOL_NUM ] = ciflag[ CCOL_NUM ] = 1;
+ oiflag[ OCOL_SUM ] = ciflag[ CCOL_SUM ] = 1;
+ oiflag[ OCOL_UPPERLIMIT_S ] = ciflag[ CCOL_UPPERLIMIT_S ] = 1;
+ ciflag[ CCOL_RIV_NUM ] = 1;
+ ciflag[ CCOL_RIV_SUM ] = 1;
break;
case UI_KEY_UPPERLIMITQUANTILE:
@@ -999,23 +1090,37 @@ columns_define_alloc(struct mkcatalogparams *p)
otype = GAL_TYPE_FLOAT32;
ctype = GAL_TYPE_FLOAT32;
disp_fmt = GAL_TABLE_DISPLAY_FMT_GENERAL;
+ disp_width = 10;
+ disp_precision = 5;
+ p->upperlimit = 1;
+ oiflag[ OCOL_UPPERLIMIT_Q ] = ciflag[ CCOL_UPPERLIMIT_Q ] = 1;
+ break;
+
+ case UI_KEY_UPPERLIMITSKEW:
+ name = "UPPERLIMIT_SKEW";
+ unit = "frac";
+ ocomment = "(Mean-Median)/STD of random distribution.";
+ ccomment = ocomment;
+ otype = GAL_TYPE_FLOAT32;
+ ctype = GAL_TYPE_FLOAT32;
+ disp_fmt = GAL_TABLE_DISPLAY_FMT_FLOAT;
disp_width = 8;
disp_precision = 3;
p->upperlimit = 1;
+ oiflag[ OCOL_UPPERLIMIT_SKEW ] = oiflag[ CCOL_UPPERLIMIT_SKEW ] = 1;
break;
case UI_KEY_RIVERAVE:
name = "RIVER_AVE";
- unit = p->input->unit ? p->input->unit : "pixelunit";
+ unit = MKCATALOG_NO_UNIT;
ocomment = NULL;
ccomment = "Average river value surrounding this clump.";
otype = GAL_TYPE_INVALID;
ctype = GAL_TYPE_FLOAT32;
disp_fmt = GAL_TABLE_DISPLAY_FMT_GENERAL;
disp_width = 10;
- disp_precision = 4;
- ciflag[ CCOL_RIV_NUM ] = 1;
- ciflag[ CCOL_RIV_SUM ] = 1;
+ disp_precision = 5;
+ ciflag[ CCOL_RIV_NUM ] = ciflag[ CCOL_RIV_SUM ] = 1;
break;
case UI_KEY_RIVERNUM:
@@ -1041,17 +1146,16 @@ columns_define_alloc(struct mkcatalogparams *p)
disp_fmt = GAL_TABLE_DISPLAY_FMT_FLOAT;
disp_width = 10;
disp_precision = 3;
- oiflag[ OCOL_SUMSTD ] = 1;
- oiflag[ OCOL_NUM ] = 1;
- oiflag[ OCOL_SUM ] = 1;
- ciflag[ CCOL_SUMSTD ] = 1;
- ciflag[ CCOL_NUM ] = 1;
- ciflag[ CCOL_SUM ] = 1;
+ oiflag[ OCOL_SUM ] = ciflag[ CCOL_SUM ] = 1;
+ oiflag[ OCOL_SUM_VAR ] = ciflag[ CCOL_SUM_VAR ] = 1;
+ ciflag[ CCOL_RIV_NUM ] = 1;
+ ciflag[ CCOL_RIV_SUM ] = 1;
+ ciflag[ CCOL_RIV_SUM_VAR ] = 1;
break;
case UI_KEY_SKY:
name = "SKY";
- unit = p->input->unit ? p->input->unit : "pixelunit";
+ unit = MKCATALOG_NO_UNIT;
ocomment = "Average input sky value.";
ccomment = ocomment;
otype = GAL_TYPE_FLOAT32;
@@ -1059,15 +1163,13 @@ columns_define_alloc(struct mkcatalogparams *p)
disp_fmt = GAL_TABLE_DISPLAY_FMT_GENERAL;
disp_width = 10;
disp_precision = 4;
- oiflag[ OCOL_NUM ] = 1;
- oiflag[ OCOL_SUMSKY ] = 1;
- ciflag[ CCOL_NUM ] = 1;
- ciflag[ CCOL_SUMSKY ] = 1;
+ oiflag[ OCOL_NUM ] = ciflag[ CCOL_NUM ] = 1;
+ oiflag[ OCOL_SUMSKY ] = ciflag[ CCOL_SUMSKY ] = 1;
break;
case UI_KEY_STD:
name = "STD";
- unit = p->input->unit ? p->input->unit : "pixelunit";
+ unit = MKCATALOG_NO_UNIT;
ocomment = "Average of input standard deviation.";
ccomment = ocomment;
otype = GAL_TYPE_FLOAT32;
@@ -1075,10 +1177,8 @@ columns_define_alloc(struct mkcatalogparams *p)
disp_fmt = GAL_TABLE_DISPLAY_FMT_GENERAL;
disp_width = 10;
disp_precision = 4;
- oiflag[ OCOL_NUM ] = 1;
- oiflag[ OCOL_SUMSTD ] = 1;
- ciflag[ CCOL_NUM ] = 1;
- ciflag[ CCOL_SUMSTD ] = 1;
+ oiflag[ OCOL_NUM ] = ciflag[ CCOL_NUM ] = 1;
+ oiflag[ OCOL_SUMSTD ] = ciflag[ CCOL_SUMSTD ] = 1;
break;
case UI_KEY_SEMIMAJOR:
@@ -1091,12 +1191,18 @@ columns_define_alloc(struct mkcatalogparams *p)
disp_fmt = GAL_TABLE_DISPLAY_FMT_FLOAT;
disp_width = 10;
disp_precision = 3;
- oiflag[ OCOL_VXX ] = 1;
- oiflag[ OCOL_VYY ] = 1;
- oiflag[ OCOL_VXY ] = 1;
- ciflag[ CCOL_VXX ] = 1;
- ciflag[ CCOL_VYY ] = 1;
- ciflag[ CCOL_VXY ] = 1;
+ oiflag[ OCOL_SUMWHT ] = ciflag[ CCOL_SUMWHT ] = 1;
+ oiflag[ OCOL_VX ] = ciflag[ CCOL_VX ] = 1;
+ oiflag[ OCOL_VY ] = ciflag[ CCOL_VY ] = 1;
+ oiflag[ OCOL_VXX ] = ciflag[ CCOL_VXX ] = 1;
+ oiflag[ OCOL_VYY ] = ciflag[ CCOL_VYY ] = 1;
+ oiflag[ OCOL_VXY ] = ciflag[ CCOL_VXY ] = 1;
+ oiflag[ OCOL_NUMALL ] = ciflag[ CCOL_NUMALL ] = 1;
+ oiflag[ OCOL_GX ] = ciflag[ CCOL_GX ] = 1;
+ oiflag[ OCOL_GY ] = ciflag[ CCOL_GY ] = 1;
+ oiflag[ OCOL_GXX ] = ciflag[ CCOL_GXX ] = 1;
+ oiflag[ OCOL_GYY ] = ciflag[ CCOL_GYY ] = 1;
+ oiflag[ OCOL_GXY ] = ciflag[ CCOL_GXY ] = 1;
break;
case UI_KEY_SEMIMINOR:
@@ -1109,12 +1215,18 @@ columns_define_alloc(struct mkcatalogparams *p)
disp_fmt = GAL_TABLE_DISPLAY_FMT_FLOAT;
disp_width = 10;
disp_precision = 3;
- oiflag[ OCOL_VXX ] = 1;
- oiflag[ OCOL_VYY ] = 1;
- oiflag[ OCOL_VXY ] = 1;
- ciflag[ CCOL_VXX ] = 1;
- ciflag[ CCOL_VYY ] = 1;
- ciflag[ CCOL_VXY ] = 1;
+ oiflag[ OCOL_SUMWHT ] = ciflag[ CCOL_SUMWHT ] = 1;
+ oiflag[ OCOL_VX ] = ciflag[ CCOL_VX ] = 1;
+ oiflag[ OCOL_VY ] = ciflag[ CCOL_VY ] = 1;
+ oiflag[ OCOL_VXX ] = ciflag[ CCOL_VXX ] = 1;
+ oiflag[ OCOL_VYY ] = ciflag[ CCOL_VYY ] = 1;
+ oiflag[ OCOL_VXY ] = ciflag[ CCOL_VXY ] = 1;
+ oiflag[ OCOL_NUMALL ] = ciflag[ CCOL_NUMALL ] = 1;
+ oiflag[ OCOL_GX ] = ciflag[ CCOL_GX ] = 1;
+ oiflag[ OCOL_GY ] = ciflag[ CCOL_GY ] = 1;
+ oiflag[ OCOL_GXX ] = ciflag[ CCOL_GXX ] = 1;
+ oiflag[ OCOL_GYY ] = ciflag[ CCOL_GYY ] = 1;
+ oiflag[ OCOL_GXY ] = ciflag[ CCOL_GXY ] = 1;
break;
case UI_KEY_AXISRATIO:
@@ -1127,12 +1239,18 @@ columns_define_alloc(struct mkcatalogparams *p)
disp_fmt = GAL_TABLE_DISPLAY_FMT_FLOAT;
disp_width = 7;
disp_precision = 3;
- oiflag[ OCOL_VXX ] = 1;
- oiflag[ OCOL_VYY ] = 1;
- oiflag[ OCOL_VXY ] = 1;
- ciflag[ CCOL_VXX ] = 1;
- ciflag[ CCOL_VYY ] = 1;
- ciflag[ CCOL_VXY ] = 1;
+ oiflag[ OCOL_SUMWHT ] = ciflag[ CCOL_SUMWHT ] = 1;
+ oiflag[ OCOL_VX ] = ciflag[ CCOL_VX ] = 1;
+ oiflag[ OCOL_VY ] = ciflag[ CCOL_VY ] = 1;
+ oiflag[ OCOL_VXX ] = ciflag[ CCOL_VXX ] = 1;
+ oiflag[ OCOL_VYY ] = ciflag[ CCOL_VYY ] = 1;
+ oiflag[ OCOL_VXY ] = ciflag[ CCOL_VXY ] = 1;
+ oiflag[ OCOL_NUMALL ] = ciflag[ CCOL_NUMALL ] = 1;
+ oiflag[ OCOL_GX ] = ciflag[ CCOL_GX ] = 1;
+ oiflag[ OCOL_GY ] = ciflag[ CCOL_GY ] = 1;
+ oiflag[ OCOL_GXX ] = ciflag[ CCOL_GXX ] = 1;
+ oiflag[ OCOL_GYY ] = ciflag[ CCOL_GYY ] = 1;
+ oiflag[ OCOL_GXY ] = ciflag[ CCOL_GXY ] = 1;
break;
case UI_KEY_POSITIONANGLE:
@@ -1145,12 +1263,18 @@ columns_define_alloc(struct mkcatalogparams *p)
disp_fmt = GAL_TABLE_DISPLAY_FMT_FLOAT;
disp_width = 10;
disp_precision = 3;
- oiflag[ OCOL_VXX ] = 1;
- oiflag[ OCOL_VYY ] = 1;
- oiflag[ OCOL_VXY ] = 1;
- ciflag[ CCOL_VXX ] = 1;
- ciflag[ CCOL_VYY ] = 1;
- ciflag[ CCOL_VXY ] = 1;
+ oiflag[ OCOL_SUMWHT ] = ciflag[ CCOL_SUMWHT ] = 1;
+ oiflag[ OCOL_VX ] = ciflag[ CCOL_VX ] = 1;
+ oiflag[ OCOL_VY ] = ciflag[ CCOL_VY ] = 1;
+ oiflag[ OCOL_VXX ] = ciflag[ CCOL_VXX ] = 1;
+ oiflag[ OCOL_VYY ] = ciflag[ CCOL_VYY ] = 1;
+ oiflag[ OCOL_VXY ] = ciflag[ CCOL_VXY ] = 1;
+ oiflag[ OCOL_NUMALL ] = ciflag[ CCOL_NUMALL ] = 1;
+ oiflag[ OCOL_GX ] = ciflag[ CCOL_GX ] = 1;
+ oiflag[ OCOL_GY ] = ciflag[ CCOL_GY ] = 1;
+ oiflag[ OCOL_GXX ] = ciflag[ CCOL_GXX ] = 1;
+ oiflag[ OCOL_GYY ] = ciflag[ CCOL_GYY ] = 1;
+ oiflag[ OCOL_GXY ] = ciflag[ CCOL_GXY ] = 1;
break;
case UI_KEY_GEOSEMIMAJOR:
@@ -1163,12 +1287,12 @@ columns_define_alloc(struct mkcatalogparams *p)
disp_fmt = GAL_TABLE_DISPLAY_FMT_FLOAT;
disp_width = 10;
disp_precision = 3;
- oiflag[ OCOL_GXX ] = 1;
- oiflag[ OCOL_GYY ] = 1;
- oiflag[ OCOL_GXY ] = 1;
- ciflag[ CCOL_GXX ] = 1;
- ciflag[ CCOL_GYY ] = 1;
- ciflag[ CCOL_GXY ] = 1;
+ oiflag[ OCOL_NUMALL ] = ciflag[ CCOL_NUMALL ] = 1;
+ oiflag[ OCOL_GX ] = ciflag[ CCOL_GX ] = 1;
+ oiflag[ OCOL_GY ] = ciflag[ CCOL_GY ] = 1;
+ oiflag[ OCOL_GXX ] = ciflag[ CCOL_GXX ] = 1;
+ oiflag[ OCOL_GYY ] = ciflag[ CCOL_GYY ] = 1;
+ oiflag[ OCOL_GXY ] = ciflag[ CCOL_GXY ] = 1;
break;
case UI_KEY_GEOSEMIMINOR:
@@ -1181,12 +1305,12 @@ columns_define_alloc(struct mkcatalogparams *p)
disp_fmt = GAL_TABLE_DISPLAY_FMT_FLOAT;
disp_width = 10;
disp_precision = 3;
- oiflag[ OCOL_GXX ] = 1;
- oiflag[ OCOL_GYY ] = 1;
- oiflag[ OCOL_GXY ] = 1;
- ciflag[ CCOL_GXX ] = 1;
- ciflag[ CCOL_GYY ] = 1;
- ciflag[ CCOL_GXY ] = 1;
+ oiflag[ OCOL_NUMALL ] = ciflag[ CCOL_NUMALL ] = 1;
+ oiflag[ OCOL_GX ] = ciflag[ CCOL_GX ] = 1;
+ oiflag[ OCOL_GY ] = ciflag[ CCOL_GY ] = 1;
+ oiflag[ OCOL_GXX ] = ciflag[ CCOL_GXX ] = 1;
+ oiflag[ OCOL_GYY ] = ciflag[ CCOL_GYY ] = 1;
+ oiflag[ OCOL_GXY ] = ciflag[ CCOL_GXY ] = 1;
break;
case UI_KEY_GEOAXISRATIO:
@@ -1199,12 +1323,12 @@ columns_define_alloc(struct mkcatalogparams *p)
disp_fmt = GAL_TABLE_DISPLAY_FMT_FLOAT;
disp_width = 7;
disp_precision = 3;
- oiflag[ OCOL_VXX ] = 1;
- oiflag[ OCOL_VYY ] = 1;
- oiflag[ OCOL_VXY ] = 1;
- ciflag[ CCOL_VXX ] = 1;
- ciflag[ CCOL_VYY ] = 1;
- ciflag[ CCOL_VXY ] = 1;
+ oiflag[ OCOL_NUMALL ] = ciflag[ CCOL_NUMALL ] = 1;
+ oiflag[ OCOL_GX ] = ciflag[ CCOL_GX ] = 1;
+ oiflag[ OCOL_GY ] = ciflag[ CCOL_GY ] = 1;
+ oiflag[ OCOL_GXX ] = ciflag[ CCOL_GXX ] = 1;
+ oiflag[ OCOL_GYY ] = ciflag[ CCOL_GYY ] = 1;
+ oiflag[ OCOL_GXY ] = ciflag[ CCOL_GXY ] = 1;
break;
case UI_KEY_GEOPOSITIONANGLE:
@@ -1217,12 +1341,12 @@ columns_define_alloc(struct mkcatalogparams *p)
disp_fmt = GAL_TABLE_DISPLAY_FMT_FLOAT;
disp_width = 10;
disp_precision = 3;
- oiflag[ OCOL_GXX ] = 1;
- oiflag[ OCOL_GYY ] = 1;
- oiflag[ OCOL_GXY ] = 1;
- ciflag[ CCOL_GXX ] = 1;
- ciflag[ CCOL_GYY ] = 1;
- ciflag[ CCOL_GXY ] = 1;
+ oiflag[ OCOL_NUMALL ] = ciflag[ CCOL_NUMALL ] = 1;
+ oiflag[ OCOL_GX ] = ciflag[ CCOL_GX ] = 1;
+ oiflag[ OCOL_GY ] = ciflag[ CCOL_GY ] = 1;
+ oiflag[ OCOL_GXX ] = ciflag[ CCOL_GXX ] = 1;
+ oiflag[ OCOL_GYY ] = ciflag[ CCOL_GYY ] = 1;
+ oiflag[ OCOL_GXY ] = ciflag[ CCOL_GXY ] = 1;
break;
default:
@@ -1252,8 +1376,7 @@ columns_define_alloc(struct mkcatalogparams *p)
allocating the column. */
if(ctype!=GAL_TYPE_INVALID)
{
- /* A clumps image has been given, so allocate space for this
- column. */
+ /* If a clumps labeled image, add this column for the output. */
if(p->clumps)
{
gal_list_data_add_alloc(&p->clumpcols, NULL, ctype, 1,
@@ -1268,23 +1391,32 @@ columns_define_alloc(struct mkcatalogparams *p)
/* If this is a clumps-only column and no clumps image was
given. Add the column to the list of similar columns to inform
- the user. */
- else if(otype==GAL_TYPE_INVALID)
+ the user.
+
+ We'll just ignore the clump-specific ID-related columns,
+ because the `--ids' (generic for both objects and clumps) is a
+ simple generic solution for identifiers. Also, ultimately,
+ they aren't measurements. */
+ else if( otype==GAL_TYPE_INVALID
+ && colcode->v!=UI_KEY_HOSTOBJID
+ && colcode->v!=UI_KEY_IDINHOSTOBJ )
gal_list_str_add(&noclumpimg, name, 1);
}
}
- /* If a warning for clumps columns and no clumps image is necessary make
- the warning. */
+ /* If the user has asked for clump-only columns, but no clumps catalog is
+ to be created (the `--clumpscat' option was not given or there were no
+ clumps in the specified image), then print an informative message that
+ the columns in question will be ignored. */
if(noclumpimg)
{
gal_list_str_reverse(&noclumpimg);
- fprintf(stderr, "\n-------\n"
- "WARNING: the following column(s) are unique to "
- "clumps (not objects), but the objects image doesn't have "
- " `WCLUMPS' keyword. So these requested columns will be "
- "ignored.\n\n");
+ fprintf(stderr, "WARNING: the following column(s) are unique to "
+ "clumps (not objects), but the `--clumpscat' option has not "
+ "been called, or there were no clumps in the clumps labeled "
+ "image. Hence, these columns will be ignored in the "
+ "output.\n\n");
for(strtmp=noclumpimg; strtmp!=NULL; strtmp=strtmp->next)
fprintf(stderr, "\t%s\n", strtmp->v);
gal_list_str_free(noclumpimg, 1);
@@ -1327,38 +1459,33 @@ columns_define_alloc(struct mkcatalogparams *p)
-/* Calculate the Signal to noise ratio for the object. */
+/* Calculate the error in brightness. */
static double
-columns_sn(struct mkcatalogparams *p, double *row, int o0c1)
+columns_brightness_error(struct mkcatalogparams *p, double *row, int o0c1)
{
- double var, sn, std, Ni, I, O;
+ double V = row[ o0c1 ? CCOL_SUM_VAR : OCOL_SUM_VAR ];
+ double OV = (o0c1 && row[ CCOL_RIV_NUM ]) ? row[ CCOL_RIV_SUM_VAR ] : 0.0;
+ return sqrt(V+OV);
+}
- /* Get all the values as averages (per pixel). */
- Ni = row[ o0c1 ? CCOL_NUM : OCOL_NUM ];
- I = MKC_RATIO( row[ o0c1 ? CCOL_SUM : OCOL_SUM ], Ni );
- std = MKC_RATIO( row[ o0c1 ? CCOL_SUMSTD : OCOL_SUMSTD ], Ni );
- var = (p->skysubtracted ? 2.0f : 1.0f) * std * std;
- /* Calculate the S/N. Note that when grown clumps are requested from
- NoiseChisel, some "clumps" will completely cover their objects and
- there will be no rivers. So if this is a clump, and the river area is
- 0, we should treat the S/N as a an object. */
- if( o0c1 && row[ CCOL_RIV_NUM ] )
- {
- /* If the Sky is already subtracted, the varience should be counted
- two times. */
- O = row[ CCOL_RIV_SUM ] / row[ CCOL_RIV_NUM ]; /* Outside. */
- sn = ( (I-O)>0
- ? ( sqrt(Ni/p->cpscorr) * (I-O)
- / sqrt( (I>0?I:-1*I) + (O>0?O:-1*O) + var ) )
- : NAN );
- }
- else
- sn = I>0 ? sqrt(Ni/p->cpscorr) * I / sqrt( (I>0?I:-1*I) + var ) : NAN;
+
+
+/* Calculate the Signal to noise ratio for the object. */
+static double
+columns_sn(struct mkcatalogparams *p, double *row, int o0c1)
+{
+ double I = row[ o0c1 ? CCOL_SUM : OCOL_SUM ];
+
+ /* When grown clumps are requested from NoiseChisel, some "clumps" will
+ completely cover their objects and there will be no rivers. So if this
+ is a clump, and the river area is 0, we should treat the S/N as a an
+ object. */
+ double O = (o0c1 && row[ CCOL_RIV_NUM ]) ? row[ CCOL_RIV_SUM ] : 0.0 ;
/* Return the derived value. */
- return sn;
+ return sqrt(1/p->cpscorr) * (I-O) / columns_brightness_error(p, row, o0c1);
}
@@ -1373,7 +1500,7 @@ columns_second_order(struct mkcatalog_passparams *pp,
double *row,
int key, int o0c1)
{
double x=NAN, y=NAN, xx=NAN, yy=NAN, xy=NAN;
- double denom, kx=pp->shift[1]+1, ky=pp->shift[0]+1;
+ double denom, kx=pp->shift[1], ky=pp->shift[0];
/* Preparations. */
switch(key)
@@ -1387,15 +1514,15 @@ columns_second_order(struct mkcatalog_passparams *pp,
double *row,
denom = row[ o0c1 ? CCOL_SUMWHT : OCOL_SUMWHT ];
/* First order. */
- x = MKC_RATIO( row[ o0c1 ? CCOL_VX : OCOL_VX ], denom );
- y = MKC_RATIO( row[ o0c1 ? CCOL_VY : OCOL_VY ], denom );
+ x = MKC_RATIO( row[ o0c1 ? CCOL_VX : OCOL_VX ], denom );
+ y = MKC_RATIO( row[ o0c1 ? CCOL_VY : OCOL_VY ], denom );
/* Second order. */
- xx = ( MKC_RATIO( row[ o0c1 ? CCOL_VXX : OCOL_VXX ], denom )
+ xx = ( MKC_RATIO( row[ o0c1 ? CCOL_VXX : OCOL_VXX ], denom )
- (x-kx) * (x-kx) );
- yy = ( MKC_RATIO( row[ o0c1 ? CCOL_VYY : OCOL_VYY ], denom )
+ yy = ( MKC_RATIO( row[ o0c1 ? CCOL_VYY : OCOL_VYY ], denom )
- (y-ky) * (y-ky) );
- xy = ( MKC_RATIO( row[ o0c1 ? CCOL_VXY : OCOL_VXY ], denom )
+ xy = ( MKC_RATIO( row[ o0c1 ? CCOL_VXY : OCOL_VXY ], denom )
- (x-kx) * (y-ky) );
break;
@@ -1408,8 +1535,8 @@ columns_second_order(struct mkcatalog_passparams *pp,
double *row,
denom = row[ o0c1 ? CCOL_NUMALL : OCOL_NUMALL ];
/* First order. */
- x = MKC_RATIO( row[ o0c1 ? CCOL_GX : OCOL_GX ], denom );
- y = MKC_RATIO( row[ o0c1 ? CCOL_GY : OCOL_GY ], denom );
+ x = MKC_RATIO( row[ o0c1 ? CCOL_GX : OCOL_GX ], denom );
+ y = MKC_RATIO( row[ o0c1 ? CCOL_GY : OCOL_GY ], denom );
/* Second order. */
xx = ( MKC_RATIO( row[ o0c1 ? CCOL_GXX : OCOL_GXX ], denom )
@@ -1438,7 +1565,6 @@ columns_second_order(struct mkcatalog_passparams *pp,
double *row,
/* Semi-minor axis. */
case UI_KEY_SEMIMINOR:
case UI_KEY_GEOSEMIMINOR:
- /*printf("\nhere\n");*/
return sqrt( ( xx + yy )/2
- sqrt( (xx - yy)/2 * (xx - yy)/2 + xy * xy ) );
@@ -1507,7 +1633,11 @@ columns_clump_brightness(double *ci)
`1/S'. So
`DM = 2.5 / ( S * ln(10) )' */
-#define MAG_ERROR(P,ROW,O0C1) (2.5f/(columns_sn((P),(ROW),(O0C1)) * log(10)))
+#define MAG_ERROR(P,ROW,O0C1) ( 2.5f \
+ / ( ( columns_sn((P),(ROW),(O0C1)) > 0 \
+ ? columns_sn((P),(ROW),(O0C1)) \
+ : NAN ) \
+ * log(10) ) )
@@ -1547,6 +1677,11 @@ columns_fill(struct mkcatalog_passparams *pp)
key=column->status;
colarr=column->array;
+ /* Put the number of clumps in the internal array which we will need
+ later to order the clump table by object ID. */
+ if(p->numclumps_c)
+ p->numclumps_c[oind]=pp->clumpsinobj;
+
/* Go over all the columns. */
switch(key)
{
@@ -1559,24 +1694,28 @@ columns_fill(struct mkcatalog_passparams *pp)
break;
case UI_KEY_AREA:
- ((int32_t *)colarr)[oind] = oi[OCOL_NUMALL];
+ ((int32_t *)colarr)[oind] = oi[OCOL_NUM];
break;
case UI_KEY_CLUMPSAREA:
- ((int32_t *)colarr)[oind] = oi[OCOL_C_NUMALL];
+ ((int32_t *)colarr)[oind] = oi[OCOL_C_NUM];
break;
case UI_KEY_WEIGHTAREA:
((int32_t *)colarr)[oind] = oi[OCOL_NUMWHT];
break;
+ case UI_KEY_GEOAREA:
+ ((int32_t *)colarr)[oind] = oi[OCOL_NUMALL];
+ break;
+
case UI_KEY_X:
- ((float *)colarr)[oind] = POS_V_G(oi, OCOL_SUMWHT, OCOL_NUMALL,
+ ((float *)colarr)[oind] = POS_V_G(oi, OCOL_SUMWHT, OCOL_NUMWHT,
OCOL_VX, OCOL_GX);
break;
case UI_KEY_Y:
- ((float *)colarr)[oind] = POS_V_G(oi, OCOL_SUMWHT, OCOL_NUMALL,
+ ((float *)colarr)[oind] = POS_V_G(oi, OCOL_SUMWHT, OCOL_NUMWHT,
OCOL_VY, OCOL_GY);
break;
@@ -1598,12 +1737,12 @@ columns_fill(struct mkcatalog_passparams *pp)
break;
case UI_KEY_CLUMPSX:
- ((float *)colarr)[oind] = POS_V_G(oi, OCOL_C_SUMWHT, OCOL_C_NUMALL,
+ ((float *)colarr)[oind] = POS_V_G(oi, OCOL_C_SUMWHT, OCOL_C_NUMWHT,
OCOL_C_VX, OCOL_C_GX);
break;
case UI_KEY_CLUMPSY:
- ((float *)colarr)[oind] = POS_V_G(oi, OCOL_C_SUMWHT, OCOL_C_NUMALL,
+ ((float *)colarr)[oind] = POS_V_G(oi, OCOL_C_SUMWHT, OCOL_C_NUMWHT,
OCOL_C_VY, OCOL_C_GY);
break;
@@ -1634,7 +1773,7 @@ columns_fill(struct mkcatalog_passparams *pp)
OCOL_GX);
vo[1][oind] = POS_V_G(oi, OCOL_SUMWHT, OCOL_NUMALL, OCOL_VY,
OCOL_GY);
- if(p->input->ndim==3)
+ if(p->objects->ndim==3)
vo[2][oind] = POS_V_G(oi, OCOL_SUMWHT, OCOL_NUMALL, OCOL_VZ,
OCOL_GZ);
break;
@@ -1644,7 +1783,7 @@ columns_fill(struct mkcatalog_passparams *pp)
case UI_KEY_GEOW3:
go[0][oind] = MKC_RATIO( oi[OCOL_GX], oi[OCOL_NUMALL] );
go[1][oind] = MKC_RATIO( oi[OCOL_GY], oi[OCOL_NUMALL] );
- if(p->input->ndim==3)
+ if(p->objects->ndim==3)
go[2][oind] = MKC_RATIO( oi[OCOL_GZ], oi[OCOL_NUMALL] );
break;
@@ -1655,7 +1794,7 @@ columns_fill(struct mkcatalog_passparams *pp)
OCOL_C_GX);
vcc[1][oind] = POS_V_G(oi, OCOL_C_SUMWHT, OCOL_C_NUMALL, OCOL_C_VY,
OCOL_C_GY);
- if(p->input->ndim==3)
+ if(p->objects->ndim==3)
vcc[2][oind] = POS_V_G(oi, OCOL_C_SUMWHT, OCOL_C_NUMALL,
OCOL_C_VZ, OCOL_C_GZ);
break;
@@ -1665,7 +1804,7 @@ columns_fill(struct mkcatalog_passparams *pp)
case UI_KEY_CLUMPSGEOW3:
gcc[0][oind] = MKC_RATIO( oi[OCOL_C_GX], oi[OCOL_C_NUMALL] );
gcc[1][oind] = MKC_RATIO( oi[OCOL_C_GY], oi[OCOL_C_NUMALL] );
- if(p->input->ndim==3)
+ if(p->objects->ndim==3)
gcc[2][oind] = MKC_RATIO( oi[OCOL_C_GZ], oi[OCOL_C_NUMALL] );
break;
@@ -1675,6 +1814,12 @@ columns_fill(struct mkcatalog_passparams *pp)
: NAN );
break;
+ case UI_KEY_BRIGHTNESSERR:
+ ((float *)colarr)[oind] = ( oi[ OCOL_NUM ]>0.0f
+ ? columns_brightness_error(p, oi, 0)
+ : NAN );
+ break;
+
case UI_KEY_CLUMPSBRIGHTNESS:
((float *)colarr)[oind] = ( oi[ OCOL_C_NUM ]>0.0f
? oi[ OCOL_C_SUM ]
@@ -1694,7 +1839,9 @@ columns_fill(struct mkcatalog_passparams *pp)
break;
case UI_KEY_MAGNITUDE:
- ((float *)colarr)[oind] = MKC_MAG(oi[ OCOL_SUM ]);
+ ((float *)colarr)[oind] = ( oi[ OCOL_NUM ]>0.0f
+ ? MKC_MAG(oi[ OCOL_SUM ])
+ : NAN );
break;
case UI_KEY_MAGNITUDEERR:
@@ -1727,6 +1874,10 @@ columns_fill(struct mkcatalog_passparams *pp)
((float *)colarr)[oind] = oi[ OCOL_UPPERLIMIT_Q ];
break;
+ case UI_KEY_UPPERLIMITSKEW:
+ ((float *)colarr)[oind] = oi[ OCOL_UPPERLIMIT_SKEW ];
+ break;
+
case UI_KEY_SN:
((float *)colarr)[oind] = columns_sn(p, oi, 0);
break;
@@ -1793,6 +1944,11 @@ columns_fill(struct mkcatalog_passparams *pp)
key = column->status;
ci = &pp->ci[ coind * CCOL_NUMCOLS ];
+ /* Put the object ID of this clump into the temporary array that we
+ will later need to sort the final clumps catalog. */
+ if(p->hostobjid_c)
+ p->hostobjid_c[cind]=pp->object;
+
/* Parse columns */
switch(key)
{
@@ -1805,13 +1961,17 @@ columns_fill(struct mkcatalog_passparams *pp)
break;
case UI_KEY_AREA:
- ((int32_t *)colarr)[cind]=ci[CCOL_NUMALL];
+ ((int32_t *)colarr)[cind]=ci[CCOL_NUM];
break;
case UI_KEY_WEIGHTAREA:
((int32_t *)colarr)[cind]=ci[CCOL_NUMWHT];
break;
+ case UI_KEY_GEOAREA:
+ ((int32_t *)colarr)[cind]=ci[CCOL_NUMALL];
+ break;
+
case UI_KEY_X:
((float *)colarr)[cind] = POS_V_G(ci, CCOL_SUMWHT, CCOL_NUMALL,
CCOL_VX, CCOL_GX);
@@ -1849,7 +2009,7 @@ columns_fill(struct mkcatalog_passparams *pp)
CCOL_GX);
vc[1][cind] = POS_V_G(ci, CCOL_SUMWHT, CCOL_NUMALL, CCOL_VY,
CCOL_GY);
- if(p->input->ndim==3)
+ if(p->objects->ndim==3)
vc[2][cind] = POS_V_G(ci, CCOL_SUMWHT, CCOL_NUMALL, CCOL_VZ,
CCOL_GZ);
break;
@@ -1859,7 +2019,7 @@ columns_fill(struct mkcatalog_passparams *pp)
case UI_KEY_GEOW3:
gc[0][cind] = MKC_RATIO( ci[CCOL_GX], ci[CCOL_NUMALL] );
gc[1][cind] = MKC_RATIO( ci[CCOL_GY], ci[CCOL_NUMALL] );
- if(p->input->ndim==3)
+ if(p->objects->ndim==3)
gc[2][cind] = MKC_RATIO( ci[CCOL_GZ], ci[CCOL_NUMALL] );
break;
@@ -1867,6 +2027,12 @@ columns_fill(struct mkcatalog_passparams *pp)
((float *)colarr)[cind] = columns_clump_brightness(ci);
break;
+ case UI_KEY_BRIGHTNESSERR:
+ ((float *)colarr)[cind] = ( ci[ CCOL_NUM ]>0.0f
+ ? columns_brightness_error(p, ci, 1)
+ : NAN );
+ break;
+
case UI_KEY_NORIVERBRIGHTNESS:
((float *)colarr)[cind] = ( ci[ CCOL_NUM ]>0.0f
? ci[ CCOL_SUM ] : NAN );
@@ -1908,11 +2074,16 @@ columns_fill(struct mkcatalog_passparams *pp)
case UI_KEY_UPPERLIMITSIGMA:
((float *)colarr)[cind] = ( columns_clump_brightness(ci)
/ ci[ CCOL_UPPERLIMIT_S ] );
+ break;
case UI_KEY_UPPERLIMITQUANTILE:
((float *)colarr)[cind] = ci[ CCOL_UPPERLIMIT_Q ];
break;
+ case UI_KEY_UPPERLIMITSKEW:
+ ((float *)colarr)[cind] = ci[ CCOL_UPPERLIMIT_SKEW ];
+ break;
+
case UI_KEY_RIVERAVE:
((float *)colarr)[cind] = ( ci[ CCOL_RIV_NUM]
? ci[ CCOL_RIV_SUM ]/ci[ CCOL_RIV_NUM]
@@ -1947,9 +2118,9 @@ columns_fill(struct mkcatalog_passparams *pp)
case UI_KEY_AXISRATIO:
((float *)colarr)[cind]
- = ( columns_second_order(pp, ci, UI_KEY_SEMIMINOR, 1)
+ = ( columns_second_order( pp, ci, UI_KEY_SEMIMINOR, 1)
/ columns_second_order(pp, ci, UI_KEY_SEMIMAJOR, 1) );
- break;
+ break;
case UI_KEY_POSITIONANGLE:
((float *)colarr)[cind] = columns_second_order(pp, ci, key, 1);
@@ -1965,9 +2136,9 @@ columns_fill(struct mkcatalog_passparams *pp)
case UI_KEY_GEOAXISRATIO:
((float *)colarr)[cind]
- = ( columns_second_order(pp, ci, UI_KEY_GEOSEMIMINOR, 1)
+ = ( columns_second_order( pp, ci, UI_KEY_GEOSEMIMINOR, 1)
/ columns_second_order(pp, ci, UI_KEY_GEOSEMIMAJOR, 1) );
- break;
+ break;
case UI_KEY_GEOPOSITIONANGLE:
((float *)colarr)[cind] = columns_second_order(pp, ci, key, 1);
diff --git a/bin/mkcatalog/main.c b/bin/mkcatalog/main.c
index 1f92630..80b16cb 100644
--- a/bin/mkcatalog/main.c
+++ b/bin/mkcatalog/main.c
@@ -45,7 +45,7 @@ main (int argc, char *argv[])
/* Read the input parameters. */
ui_read_check_inputs_setup(argc, argv, &p);
- /* Run MakeProfiles */
+ /* Run MakeCatalog */
mkcatalog(&p);
/* Free all non-freed allocations. */
diff --git a/bin/mkcatalog/main.h b/bin/mkcatalog/main.h
index 632dd64..3f6b0d8 100644
--- a/bin/mkcatalog/main.h
+++ b/bin/mkcatalog/main.h
@@ -41,6 +41,10 @@ along with Gnuastro. If not, see
<http://www.gnu.org/licenses/>.
#define MKCATALOG_UPPERLIMIT_MINIMUM_NUM 20
+/* Unit string to use if values dataset doesn't have any. */
+#define MKCATALOG_NO_UNIT "input-units"
+
+
/* Intermediate/raw array elements
===============================
@@ -67,15 +71,13 @@ along with Gnuastro. If not, see
<http://www.gnu.org/licenses/>.
enum objectcols
{
OCOL_NUMALL, /* Number of all pixels with this label. */
- OCOL_NUM, /* Number of pixels above threshold. */
+ OCOL_NUM, /* Number of pixels with a value. */
OCOL_SUM, /* Sum of (value-sky) in object. */
+ OCOL_SUM_VAR, /* Varience of sum (for brightness error). */
OCOL_MEDIAN, /* Median of value in object. */
OCOL_VX, /* Sum of (value-sky) * x. */
OCOL_VY, /* Sum of (value-sky) * y. */
OCOL_VZ, /* Sum of (value-sky) * z. */
- OCOL_SX, /* Shift along X axis. */
- OCOL_SY, /* Shift along Y axis. */
- OCOL_SZ, /* Shift along Z axis. */
OCOL_VXX, /* Sum of (value-sky) * x * x. */
OCOL_VYY, /* Sum of (value-sky) * y * y. */
OCOL_VXY, /* Sum of (value-sky) * x * y. */
@@ -92,6 +94,7 @@ enum objectcols
OCOL_UPPERLIMIT_B, /* Upper limit brightness. */
OCOL_UPPERLIMIT_S, /* Upper limit one-sigma value. */
OCOL_UPPERLIMIT_Q, /* Quantile of object in random distribution.*/
+ OCOL_UPPERLIMIT_SKEW,/* (Mean-Median)/STD of random distribution. */
OCOL_C_NUMALL, /* Value independent no. of pixels in clumps.*/
OCOL_C_NUM, /* Area of clumps in this object. */
OCOL_C_SUM, /* Brightness in object clumps. */
@@ -109,18 +112,20 @@ enum objectcols
enum clumpcols
{
- CCOL_NUMALL, /* Area of clump irrespective of threshold. */
- CCOL_NUM, /* Area of this clump. */
+ CCOL_NUMALL, /* Number of pixels in clump. */
+ CCOL_NUM, /* Number of values used in clump. */
+ CCOL_SUM, /* River subtracted brightness. */
+ CCOL_SUM_VAR, /* Variance of sum (for brightness error). */
+ CCOL_MEDIAN, /* Median of values in clump. */
+ CCOL_RIV_NUM, /* Num river pixels around this clump. */
+ CCOL_RIV_SUM, /* Sum of rivers around clump. */
+ CCOL_RIV_SUM_VAR, /* Variance of sum (for error measurements). */
CCOL_VX, /* Sum of (value-sky) * x. */
CCOL_VY, /* Sum of (value-sky) * y. */
CCOL_VZ, /* Sum of (value-sky) * z. */
CCOL_VXX, /* Sum of flux*x*x of this clump. */
CCOL_VYY, /* Sum of flux*y*y of this clump. */
CCOL_VXY, /* Sum of flux*x*y of this clump. */
- CCOL_SUM, /* River subtracted brightness. */
- CCOL_MEDIAN, /* Median of values in clump. */
- CCOL_RIV_SUM, /* Sum of rivers around clump. */
- CCOL_RIV_NUM, /* Num river pixels around this clump. */
CCOL_SUMSKY, /* Sum of sky value on this object. */
CCOL_SUMSTD, /* Sum of sky STD value on this object. */
CCOL_SUMWHT, /* Sum of positive image pixels for wht. */
@@ -134,6 +139,7 @@ enum clumpcols
CCOL_UPPERLIMIT_B, /* Upper limit brightness. */
CCOL_UPPERLIMIT_S, /* Upper limit one-sigma value. */
CCOL_UPPERLIMIT_Q, /* Quantile of object in random distribution.*/
+ CCOL_UPPERLIMIT_SKEW,/* (Mean-Median)/STD of random distribution. */
CCOL_NUMCOLS, /* SHOULD BE LAST: total number of columns. */
};
@@ -150,9 +156,9 @@ struct mkcatalogparams
/* From command-line */
struct gal_options_common_params cp; /* Common parameters. */
gal_list_i32_t *columnids; /* The desired column codes. */
- char *inputname; /* Input filename. */
- char *objectsfile; /* File name of objects file. */
- char *objectshdu; /* HDU of objects image. */
+ char *objectsfile; /* Input filename. */
+ char *valuesfile; /* File name of objects file. */
+ char *valueshdu; /* HDU of objects image. */
char *clumpsfile; /* File name of objects file. */
char *clumpshdu; /* HDU of objects image. */
char *skyfile; /* File name of sky file. */
@@ -160,32 +166,33 @@ struct mkcatalogparams
char *stdfile; /* File name of sky STD file. */
char *stdhdu; /* HDU of sky STD image. */
+ uint8_t clumpscat; /* ==1: create clumps catalog. */
+ uint8_t noclumpsort; /* Don't sort the clumps catalog. */
float zeropoint; /* Zero-point magnitude of object. */
- uint8_t skysubtracted; /* If image is already sky subtracted. */
- float threshold; /* Only use values above this threshold.*/
+ uint8_t variance; /* Input STD file is actually variance. */
+ uint8_t subtractsky; /* ==1: subtract the Sky from values. */
float sfmagnsigma; /* Surface brightness multiple of sigma.*/
float sfmagarea; /* Surface brightness area (arcsec^2). */
char *upmaskfile; /* Name of upper limit mask file. */
char *upmaskhdu; /* HDU of upper limit mask file. */
size_t upnum; /* Number of upper-limit random samples.*/
- size_t *uprange; /* Range of random positions about target.*/
+ size_t *uprange; /* Range of random pos. around target. */
uint8_t envseed; /* Use the environment for random seed. */
double upsigmaclip[2]; /* Sigma clip to measure upper limit. */
float upnsigma; /* Multiple of sigma to define up-lim. */
+ int32_t checkupperlimit[2]; /* Object & clump ID to check dist. */
/* Internal. */
time_t rawtime; /* Starting time of the program. */
- gal_data_t *input; /* Input. */
+ gal_data_t *values; /* Input. */
gal_data_t *objects; /* Object labels. */
gal_data_t *clumps; /* Clump labels. */
gal_data_t *sky; /* Sky. */
gal_data_t *std; /* Sky standard deviation. */
gal_data_t *upmask; /* Upper limit magnitude mask. */
- float minstd; /* Minimum Standard deviation value. */
float medstd; /* Median standard deviation value. */
float cpscorr; /* Counts-per-second correction. */
- float detsn; /* Minimum detection S/N threshold. */
size_t numobjects; /* Number of object labels in image. */
float clumpsn; /* Clump S/N threshold. */
size_t numclumps; /* Number of clumps in image. */
@@ -194,6 +201,7 @@ struct mkcatalogparams
gal_data_t *tiles; /* Tiles to cover each object. */
char *objectsout; /* Output objects catalog. */
char *clumpsout; /* Output clumps catalog. */
+ char *upcheckout; /* Name of upperlimit check table. */
uint8_t *oiflag; /* Intermediate flags for objects. */
uint8_t *ciflag; /* Intermediate flags for clumps. */
pthread_mutex_t mutex; /* Mutex to change the total numbers. */
@@ -203,6 +211,14 @@ struct mkcatalogparams
const char *rngname; /* Name of random number generator. */
size_t rngmin; /* Minimum possible value of RNG. */
size_t rngdiff; /* Difference of RNG max and min. */
+ uint8_t uprangewarning; /* A warning must be printed. */
+ size_t *hostobjid_c; /* To sort the clumps table by Obj.ID. */
+ size_t *numclumps_c; /* To sort the clumps table by Obj.ID. */
+
+ char *usedvaluesfile; /* Ptr to final name used for values. */
+ char *usedclumpsfile; /* Ptr to final name used for clumps. */
+ char *usedskyfile; /* Ptr to final fname used for sky. */
+ char *usedstdfile; /* Ptr to final name used for sky std. */
gal_data_t *wcs_vo; /* Object RA-Dec flux weighted X, Y. */
gal_data_t *wcs_vc; /* Clump RA-Dec flux weighted X, Y. */
diff --git a/bin/mkcatalog/mkcatalog.c b/bin/mkcatalog/mkcatalog.c
index e09929c..683b0e6 100644
--- a/bin/mkcatalog/mkcatalog.c
+++ b/bin/mkcatalog/mkcatalog.c
@@ -38,6 +38,7 @@ along with Gnuastro. If not, see
<http://www.gnu.org/licenses/>.
#include <gnuastro/threads.h>
#include <gnuastro/dimension.h>
#include <gnuastro/statistics.h>
+#include <gnuastro/permutation.h>
#include <gnuastro-internal/timing.h>
@@ -45,6 +46,7 @@ along with Gnuastro. If not, see
<http://www.gnu.org/licenses/>.
#include "mkcatalog.h"
#include "ui.h"
+#include "parse.h"
#include "columns.h"
#include "upperlimit.h"
@@ -52,508 +54,13 @@ along with Gnuastro. If not, see
<http://www.gnu.org/licenses/>.
-/*********************************************************************/
-/************* Definitions and initialization ***************/
-/*********************************************************************/
-/* Both passes are going to need their starting pointers set, so we'll do
- that here. */
-static void
-mkcatalog_initialize_params(struct mkcatalog_passparams *pp)
-{
- struct mkcatalogparams *p=pp->p;
-
- /* Initialize the number of clumps in this object. */
- pp->clumpsinobj=0;
-
-
- /* Initialize the intermediate values. */
- memset(pp->oi, 0, OCOL_NUMCOLS * sizeof *pp->oi);
-
-
- /* Set the shifts in every dimension to avoid round-off errors in large
- numbers for the non-linear calculations. We are using the first pixel
- of each object's tile as the shift parameter to keep the mean
- (average) reasonably near to the standard deviation. Otherwise, when
- the object is far out in the image (large x and y positions), then
- roundoff errors are going to decrease the accuracy of the second order
- calculations. */
- gal_dimension_index_to_coord( ( (float *)(pp->tile->array)
- - (float *)(pp->tile->block->array) ),
- p->input->ndim, p->input->dsize, pp->shift);
-
-
- /* Set the starting and ending indexs of this tile/object. */
- pp->st_i = gal_tile_start_end_ind_inclusive(pp->tile, p->input,
- pp->start_end_inc);
- pp->st_sky = (float *)(p->sky->array) + pp->start_end_inc[0];
- pp->st_std = (float *)(p->std->array) + pp->start_end_inc[0];
- pp->st_o = (int32_t *)(p->objects->array) + pp->start_end_inc[0];
- pp->st_c = ( p->clumps
- ? (int32_t *)(p->clumps->array) + pp->start_end_inc[0]
- : NULL );
-}
-
-
-
-
-
-
-
-
-
-
-
-
-
-
-
/*********************************************************************/
-/************* First and second passes ***************/
-/*********************************************************************/
-static void
-mkcatalog_first_pass(struct mkcatalog_passparams *pp)
-{
- struct mkcatalogparams *p=pp->p;
- size_t ndim=p->input->ndim, *dsize=p->input->dsize;
-
- double *oi=pp->oi;
- int32_t *O, *C=NULL;
- size_t d, increment=0, num_increment=1;
- float ss, *I, *II, *SK, *ST, *input=p->input->array;
- size_t *c=gal_data_malloc_array(GAL_TYPE_SIZE_T, ndim, __func__, "c");
- size_t *sc=gal_data_malloc_array(GAL_TYPE_SIZE_T, ndim, __func__, "sc");
-
-
- /* Parse each contiguous patch of memory covered by this object. */
- while( pp->start_end_inc[0] + increment <= pp->start_end_inc[1] )
- {
- /* Set the contiguous range to parse, we will check the count
- over the `I' pointer and just increment the rest. */
- O = pp->st_o + increment;
- SK = pp->st_sky + increment;
- ST = pp->st_std + increment;
- if(p->clumps) C = pp->st_c + increment;
- II = ( I = pp->st_i + increment ) + pp->tile->dsize[ndim-1];
-
- /* Parse the tile. */
- do
- {
- /* If this pixel belongs to the requested object then do the
- processing. */
- if( *O==pp->object )
- {
- /* Get the number of clumps in this object: the largest clump
- ID over each object. */
- if( p->clumps && *C>0 )
- pp->clumpsinobj = *C > pp->clumpsinobj ? *C : pp->clumpsinobj;
-
-
- /* Get the coordinates of this point. */
- gal_dimension_index_to_coord(I-input, ndim, dsize, c);
-
-
- /* Calculate the shifted coordinates for second order
- calculations. The coordinate is incremented because from
- now on, the positions are in the FITS standard (starting
- from one).
-
- IMPORTANT NOTE: this is a postfix increment, so after the
- expression (difference) is evaluated, the coordinate is
- going to change. This is necessary because `shift' is also
- starting from zero. */
- for(d=0;d<ndim;++d) sc[d] = c[d]++ - pp->shift[d];
-
-
- /* Do the general geometric (independent of pixel value)
- calculations. */
- oi[ OCOL_NUMALL ]++;
- oi[ OCOL_GX ] += c[ndim-1];
- oi[ OCOL_GY ] += c[ndim-2];
- if(ndim==3)
- oi[ OCOL_GZ ] += c[ndim-3];
- oi[ OCOL_GXX ] += sc[1] * sc[1];
- oi[ OCOL_GYY ] += sc[0] * sc[0];
- oi[ OCOL_GXY ] += sc[1] * sc[0];
- if(p->clumps && *C>0)
- {
- oi[ OCOL_C_GX ] += c[ndim-1];
- oi[ OCOL_C_GY ] += c[ndim-2];
- if(ndim==3)
- oi[ OCOL_C_GZ ] += c[ndim-3];
- }
-
-
- /* Start the pixel value related parameters.
-
- ABOUT THE CHECK: The reason this condition is given
- like this is that the `threshold' value is optional
- and we don't want to do multiple checks.
-
- The basic idea is this: when the user doesn't want any
- thresholds applied, then `p->threshold==NAN' and any
- conditional that involves a NaN will fail, so its logical
- negation will be positive and the calculations below will
- be done. However, if the user does specify a threhold and
- the pixel is above the threshold, then (`ss < p->threshold
- * *ST') will be false and its logical negation will be
- positive, so the pixel will be included. */
- if( !( p->hasblank && isnan(*I) )
- && !( (ss = *I - *SK) < p->threshold * *ST ) )
- {
- /* General flux summations. */
- oi[ OCOL_NUM ]++;
- oi[ OCOL_SUM ] += ss;
- oi[ OCOL_SUMSKY ] += *SK;
- oi[ OCOL_SUMSTD ] += *ST;
- if(p->clumps && *C>0)
- {
- oi[ OCOL_C_NUM ]++;
- oi[ OCOL_C_SUM ] += ss;
- }
-
- /* For flux weighted centers, we can only use positive
- values, so do those measurements here. */
- if( ss > 0.0f )
- {
- oi[ OCOL_NUMWHT ]++;
- oi[ OCOL_SUMWHT ] += ss;
- oi[ OCOL_VX ] += ss * c[ndim-1];
- oi[ OCOL_VY ] += ss * c[ndim-2];
- if(ndim==3)
- oi[ OCOL_VZ ] += ss * c[ndim-3];
- oi[ OCOL_VXX ] += ss * sc[1] * sc[1];
- oi[ OCOL_VYY ] += ss * sc[0] * sc[0];
- oi[ OCOL_VXY ] += ss * sc[1] * sc[0];
- if(p->clumps && *C>0)
- {
- oi[ OCOL_C_NUMWHT ]++;
- oi[ OCOL_C_SUMWHT ] += ss;
- oi[ OCOL_C_VX ] += ss * c[ndim-1];
- oi[ OCOL_C_VY ] += ss * c[ndim-2];
- if(ndim==3)
- oi[ OCOL_C_VZ ] += ss * c[ndim-3];
- }
- }
- }
- }
-
- /* Increment the other pointers. */
- ++O; ++SK; ++ST; if(p->clumps) ++C;
- }
- while(++I<II);
-
- /* Increment to the next contiguous region of this tile. */
- increment += ( gal_tile_block_increment(p->input, dsize,
- num_increment++, NULL) );
- }
-
- /* Clean up. */
- free(c);
- free(sc);
-}
-
-
-
-
-
-/* Do the second pass */
-static void
-mkcatalog_second_pass(struct mkcatalog_passparams *pp)
-{
- struct mkcatalogparams *p=pp->p;
- size_t ndim=p->input->ndim, *dsize=p->input->dsize;
-
- double *ci;
- int32_t *O, *C=NULL, nlab, *ngblabs;
- size_t i, ii, d, increment=0, num_increment=1;
- size_t nngb=gal_dimension_num_neighbors(ndim);
- size_t *dinc=gal_dimension_increment(ndim, dsize);
- float ss, *I, *II, *SK, *ST, *input=p->input->array;
- int32_t *objects=p->objects->array, *clumps=p->clumps->array;
- size_t *c=gal_data_malloc_array(GAL_TYPE_SIZE_T, ndim, __func__, "c");
- size_t *sc=gal_data_malloc_array(GAL_TYPE_SIZE_T, ndim, __func__, "sc");
-
- /* Allocate array to keep the neighbor labels. */
- ngblabs=gal_data_malloc_array(GAL_TYPE_INT32, nngb, __func__, "ngblabs");
-
- /* Parse each contiguous patch of memory covered by this object. */
- while( pp->start_end_inc[0] + increment <= pp->start_end_inc[1] )
- {
- /* Set the contiguous range to parse, we will check the count
- over the `I' pointer and just increment the rest. */
- O = pp->st_o + increment;
- SK = pp->st_sky + increment;
- ST = pp->st_std + increment;
- if(p->clumps) C = pp->st_c + increment;
- II = ( I = pp->st_i + increment ) + pp->tile->dsize[ndim-1];
-
- /* Parse the next contiguous region of this tile. */
- do
- {
- /* If this pixel belongs to the requested object, is a clumps and
- isn't NAN, then do the processing. `hasblank' is constant, so
- when the input doesn't have any blank values, the `isnan' will
- never be checked. */
- if( *O==pp->object )
- {
- /* We are on a clump. */
- if(p->clumps && *C>0)
- {
- /* Pointer to make things easier. Note that the clump
- labels start from 1, but the array indexs from 0.*/
- ci=&pp->ci[ (*C-1) * CCOL_NUMCOLS ];
-
- /* Get the coordinates of this point. */
- gal_dimension_index_to_coord(I-input, ndim, dsize, c);
-
- /* Shifted coordinates for second order moments, see
- explanations in the first pass.*/
- for(d=0;d<ndim;++d) sc[d] = c[d]++ - pp->shift[d];
-
- /* Geometric measurements (independent of pixel value). */
- ci[ CCOL_NUMALL ]++;
- ci[ CCOL_GX ] += c[ndim-1];
- ci[ CCOL_GY ] += c[ndim-2];
- if(ndim==3)
- ci[ CCOL_GZ ] += c[ndim-3];
- ci[ CCOL_GXX ] += sc[1] * sc[1];
- ci[ CCOL_GYY ] += sc[0] * sc[0];
- ci[ CCOL_GXY ] += sc[1] * sc[0];
-
- /* Only use pixels above the threshold, see explanations in
- first pass for an explanation. */
- if( !( p->hasblank && isnan(*I) )
- && !( (ss = *I - *SK) < p->threshold * *ST ) )
- {
- /* Fill in the necessary information. */
- ci[ CCOL_NUM ]++;
- ci[ CCOL_SUM ] += ss;
- ci[ CCOL_SUMSKY ] += *SK;
- ci[ CCOL_SUMSTD ] += *ST;
- if( ss > 0.0f )
- {
- ci[ CCOL_NUMWHT ]++;
- ci[ CCOL_SUMWHT ] += ss;
- ci[ CCOL_VX ] += ss * c[ndim-1];
- ci[ CCOL_VY ] += ss * c[ndim-2];
- if(ndim==3)
- ci[ CCOL_VZ ] += ss * c[ndim-3];
- ci[ CCOL_VXX ] += ss * sc[1] * sc[1];
- ci[ CCOL_VYY ] += ss * sc[0] * sc[0];
- ci[ CCOL_VXY ] += ss * sc[1] * sc[0];
- }
- }
- }
-
- /* This pixel is on the diffuse region, check to see if it is
- touching a clump or not, but only if this object actually
- has any clumps. */
- else if(pp->clumpsinobj)
- {
- /* We are on a diffuse (possibly a river) pixel. So the
- value of this pixel has to be added to any of the
- clumps in touches. But since it might touch a labeled
- region more than once, we use `ngblabs' to keep track
- of which label we have already added its value
- to. `ii' is the number of different labels this river
- pixel has already been considered for. `ngblabs' will
- keep the list labels. */
- ii=0;
- memset(ngblabs, 0, nngb*sizeof *ngblabs);
-
- /* Go over the neighbors and see if this pixel is
- touching a clump or not. */
- GAL_DIMENSION_NEIGHBOR_OP(I-input, ndim, dsize, ndim, dinc,
- {
- /* Neighbor's label (mainly for easy reading). */
- nlab=clumps[nind];
-
- /* We only want neighbors that are a clump and part
- of this object and part of the same object. */
- if( nlab>0 && objects[nind]==pp->object)
- {
- /* Go over all already checked labels and make
- sure this clump hasn't already been
- considered. */
- for(i=0;i<ii;++i) if(ngblabs[i]==nlab) break;
-
- /* It hasn't been considered yet: */
- if(i==ii)
- {
- ngblabs[ii++] = nlab;
-
- ++(pp->ci)[ (nlab-1) * CCOL_NUMCOLS
- + CCOL_RIV_NUM ];
- pp->ci[ (nlab-1) * CCOL_NUMCOLS
- + CCOL_RIV_SUM ] += *I-*SK;
- }
- }
- });
- }
- }
-
- /* Increment the other pointers. */
- ++O; ++SK; ++ST; if(p->clumps) ++C;
- }
- while(++I<II);
-
- /* Increment to the next contiguous region of this tile. */
- increment += ( gal_tile_block_increment(p->input, dsize,
- num_increment++, NULL) );
- }
-
- /* Clean up. */
- free(c);
- free(sc);
- free(dinc);
- free(ngblabs);
-}
-
-
-
-
-
-static void
-mkcatalog_median_pass(struct mkcatalog_passparams *pp)
-{
- struct mkcatalogparams *p=pp->p;
- size_t ndim=p->input->ndim, *dsize=p->input->dsize;
-
- double *ci;
- gal_data_t *median;
- int32_t *O, *C=NULL;
- gal_data_t **clumpsmed=NULL;
- float ss, *I, *II, *SK, *ST;
- size_t i, increment=0, num_increment=1;
- size_t counter=0, *ccounter=NULL, tsize=pp->oi[OCOL_NUM];
- gal_data_t *objmed=gal_data_alloc(NULL, p->input->type, 1, &tsize, NULL, 0,
- p->cp.minmapsize, NULL, NULL, NULL);
-
- /* A small sanity check. */
- if(p->input->type!=GAL_TYPE_FLOAT32)
- error(EXIT_FAILURE, 0, "%s: the input has to be float32 type", __func__);
-
-
- /* Allocate space for the clump medians. */
- if(p->clumps)
- {
- errno=0;
- clumpsmed=malloc(pp->clumpsinobj * sizeof *clumpsmed);
- if(clumpsmed==NULL)
- error(EXIT_FAILURE, errno, "%s: couldn't allocate `clumpsmed' for "
- "%zu clumps", __func__, pp->clumpsinobj);
-
-
- /* Allocate the array necessary to keep the values of each clump. */
- ccounter=gal_data_calloc_array(GAL_TYPE_SIZE_T, pp->clumpsinobj,
- __func__, "ccounter");
- for(i=0;i<pp->clumpsinobj;++i)
- {
- tsize=pp->ci[ i * CCOL_NUMCOLS + CCOL_NUM ];
- clumpsmed[i]=gal_data_alloc(NULL, p->input->type, 1, &tsize, NULL,
- 0, p->cp.minmapsize, NULL, NULL, NULL);
- }
- }
-
-
- /* Parse each contiguous patch of memory covered by this object. */
- while( pp->start_end_inc[0] + increment <= pp->start_end_inc[1] )
- {
- /* Set the contiguous range to parse, we will check the count
- over the `I' pointer and just increment the rest. */
- O = pp->st_o + increment;
- SK = pp->st_sky + increment;
- ST = pp->st_std + increment;
- if(p->clumps) C = pp->st_c + increment;
- II = ( I = pp->st_i + increment ) + pp->tile->dsize[ndim-1];
-
- /* Parse the next contiguous region of this tile. */
- do
- {
- /* If this pixel belongs to the requested object, then do the
- processing. `hasblank' is constant, so when the input doesn't
- have any blank values, the `isnan' will never be checked. */
- if( *O==pp->object
- && !( p->hasblank && isnan(*I) )
- && !( (ss = *I - *SK) < p->threshold * *ST ))
- {
- /* Copy the value for the whole object. */
- ss = *I - *SK;
- memcpy( gal_data_ptr_increment(objmed->array, counter++,
- p->input->type),
- &ss, gal_type_sizeof(p->input->type) );
-
- /* We are also on a clump. */
- if(p->clumps && *C>0)
- memcpy( gal_data_ptr_increment(clumpsmed[*C-1]->array,
- ccounter[*C-1]++,
- p->input->type),
- &ss, gal_type_sizeof(p->input->type) );
- }
-
- /* Increment the other pointers. */
- ++O; ++SK; ++ST; if(p->clumps) ++C;
- }
- while(++I<II);
-
- /* Increment to the next contiguous region of this tile. */
- increment += ( gal_tile_block_increment(p->input, dsize,
- num_increment++, NULL) );
- }
-
-
- /* Calculate the final medians for objects. */
- median=gal_data_copy_to_new_type_free(gal_statistics_median(objmed, 1),
- GAL_TYPE_FLOAT64);
- pp->oi[OCOL_MEDIAN]=*((double *)(median->array));
- gal_data_free(objmed);
- gal_data_free(median);
-
-
- /* Calculate the median for clumps. */
- if(p->clumps)
- {
- for(i=0;i<pp->clumpsinobj;++i)
- {
- ci=&pp->ci[ i * CCOL_NUMCOLS ];
- median=gal_statistics_median(clumpsmed[i], 1);
- median=gal_data_copy_to_new_type_free(median, GAL_TYPE_FLOAT64);
- ci[ CCOL_MEDIAN ] = ( *((double *)(median->array))
- - (ci[ CCOL_RIV_SUM ]/ci[ CCOL_RIV_NUM ]) );
- gal_data_free(clumpsmed[i]);
- gal_data_free(median);
- }
- free(clumpsmed);
- free(ccounter);
- }
-}
-
-
-
-
-
-
-
-
-
-
-
-
-
-
-
-
-
-
-
-
-/*********************************************************************/
-/***************** High-level funcitons *******************/
+/************** Manage a single object *******************/
/*********************************************************************/
static void
mkcatalog_clump_starting_index(struct mkcatalog_passparams *pp)
@@ -587,41 +94,56 @@ mkcatalog_single_object(void *in_prm)
{
struct gal_threads_params *tprm=(struct gal_threads_params *)in_prm;
struct mkcatalogparams *p=(struct mkcatalogparams *)(tprm->params);
- size_t ndim=p->input->ndim;
+ size_t ndim=p->objects->ndim;
size_t i;
+ uint8_t *oif=p->oiflag;
struct mkcatalog_passparams pp;
+
/* Initialize the mkcatalog_passparams elements. */
pp.p = p;
pp.clumpstartindex = 0;
pp.rng = p->rng ? gsl_rng_clone(p->rng) : NULL;
- pp.shift = gal_data_malloc_array(GAL_TYPE_SIZE_T, ndim, __func__,
- "pp.shift");
pp.oi = gal_data_malloc_array(GAL_TYPE_FLOAT64, OCOL_NUMCOLS,
__func__, "pp.oi");
+ /* If we have second order measurements, allocate the array keeping the
+ temporary shift values for each object of this thread. Note that the
+ clumps catalog (if requested), will have the same measurements, so its
+ just enough to check the objects. */
+ pp.shift = ( ( oif[ OCOL_GXX ]
+ || oif[ OCOL_GYY ]
+ || oif[ OCOL_GXY ]
+ || oif[ OCOL_VXX ]
+ || oif[ OCOL_VYY ]
+ || oif[ OCOL_VXY ] )
+ ? gal_data_malloc_array(GAL_TYPE_SIZE_T, ndim, __func__,
+ "pp.shift")
+ : NULL );
+
/* If we have upper-limit mode, then allocate the container to keep the
values to calculate the standard deviation. */
- pp.up_vals = p->upperlimit ? gal_data_alloc(NULL, GAL_TYPE_FLOAT32, 1,
- &p->upnum, NULL, 0,
- p->cp.minmapsize, NULL, NULL,
- NULL) : NULL;
+ pp.up_vals = ( p->upperlimit
+ ? gal_data_alloc(NULL, GAL_TYPE_FLOAT32, 1, &p->upnum,
+ NULL, 0, p->cp.minmapsize, NULL, NULL,
+ NULL)
+ : NULL );
/* Fill the desired columns for all the objects given to this thread. */
for(i=0; tprm->indexs[i]!=GAL_BLANK_SIZE_T; ++i)
{
- /* For easy reading, Note that the object IDs start from one while
+ /* For easy reading. Note that the object IDs start from one while
the array positions start from 0. */
pp.ci=NULL;
pp.object = tprm->indexs[i] + 1;
pp.tile = &p->tiles[ tprm->indexs[i] ];
/* Initialize the parameters for this object/tile. */
- mkcatalog_initialize_params(&pp);
+ parse_initialize(&pp);
/* Get the first pass information. */
- mkcatalog_first_pass(&pp);
+ parse_objects(&pp);
/* Currently the second pass is only necessary when there is a clumps
image. */
@@ -638,12 +160,12 @@ mkcatalog_single_object(void *in_prm)
mkcatalog_clump_starting_index(&pp);
/* Get the second pass information. */
- mkcatalog_second_pass(&pp);
+ parse_clumps(&pp);
}
/* If the median is requested, another pass is necessary. */
if( p->oiflag[ OCOL_MEDIAN ] )
- mkcatalog_median_pass(&pp);
+ parse_median(&pp);
/* Calculate the upper limit magnitude (if necessary). */
if(p->upperlimit) upperlimit_calculate(&pp);
@@ -701,29 +223,29 @@ mkcatalog_wcs_conversion(struct mkcatalogparams *p)
/* Flux weighted center positions for clumps and objects. */
if(p->wcs_vo)
{
- gal_wcs_img_to_world(p->wcs_vo, p->input->wcs, 1);
+ gal_wcs_img_to_world(p->wcs_vo, p->objects->wcs, 1);
if(p->wcs_vc)
- gal_wcs_img_to_world(p->wcs_vc, p->input->wcs, 1);
+ gal_wcs_img_to_world(p->wcs_vc, p->objects->wcs, 1);
}
/* Geometric center positions for clumps and objects. */
if(p->wcs_go)
{
- gal_wcs_img_to_world(p->wcs_go, p->input->wcs, 1);
+ gal_wcs_img_to_world(p->wcs_go, p->objects->wcs, 1);
if(p->wcs_gc)
- gal_wcs_img_to_world(p->wcs_gc, p->input->wcs, 1);
+ gal_wcs_img_to_world(p->wcs_gc, p->objects->wcs, 1);
}
/* All clumps flux weighted center. */
if(p->wcs_vcc)
- gal_wcs_img_to_world(p->wcs_vcc, p->input->wcs, 1);
+ gal_wcs_img_to_world(p->wcs_vcc, p->objects->wcs, 1);
/* All clumps geometric center. */
if(p->wcs_gcc)
- gal_wcs_img_to_world(p->wcs_gcc, p->input->wcs, 1);
+ gal_wcs_img_to_world(p->wcs_gcc, p->objects->wcs, 1);
/* Go over all the object columns and fill in the values. */
@@ -788,19 +310,102 @@ mkcatalog_wcs_conversion(struct mkcatalogparams *p)
+void
+mkcatalog_write_inputs_in_comments(struct mkcatalogparams *p,
+ gal_list_str_t **comments, int withsky,
+ int withstd)
+{
+ char *tmp, *str;
+
+ /* Basic classifiers for plain text outputs. */
+ if(p->cp.tableformat==GAL_TABLE_FORMAT_TXT)
+ {
+ if( asprintf(&str, "--------- Input files ---------")<0 )
+ error(EXIT_FAILURE, 0, "%s: asprintf allocation", __func__);
+ gal_list_str_add(comments, str, 0);
+ }
+
+ /* Object labels. */
+ if( asprintf(&str, "Objects: %s (hdu: %s).", p->objectsfile, p->cp.hdu)<0 )
+ error(EXIT_FAILURE, 0, "%s: asprintf allocation", __func__);
+ gal_list_str_add(comments, str, 0);
+
+ /* Clump labels. */
+ if(p->clumps)
+ {
+ if(asprintf(&str, "Clumps: %s (hdu: %s).", p->usedclumpsfile,
+ p->clumpshdu)<0)
+ error(EXIT_FAILURE, 0, "%s: asprintf allocation", __func__);
+ gal_list_str_add(comments, str, 0);
+ }
+
+ /* Values dataset. */
+ if(p->values)
+ {
+ if( asprintf(&str, "Values: %s (hdu: %s).", p->usedvaluesfile,
+ p->valueshdu)<0 )
+ error(EXIT_FAILURE, 0, "%s: asprintf allocation", __func__);
+ gal_list_str_add(comments, str, 0);
+ }
+
+ /* Sky dataset. */
+ if(withsky && p->sky)
+ {
+ if(p->sky->size==1)
+ {
+ if( asprintf(&str, "Sky: %g.", *((float *)(p->sky->array)) )<0 )
+ error(EXIT_FAILURE, 0, "%s: asprintf allocation", __func__);
+ }
+ else
+ {
+ if( asprintf(&str, "Sky: %s (hdu: %s).", p->usedskyfile,
+ p->skyhdu)<0 )
+ error(EXIT_FAILURE, 0, "%s: asprintf allocation", __func__);
+ }
+ gal_list_str_add(comments, str, 0);
+ }
+
+ /* Sky standard deviation dataset. */
+ tmp = p->variance ? "VAR" : "STD";
+ if(withstd && p->std)
+ {
+ if(p->std->size==1)
+ {
+ if( asprintf(&str, "Sky %s: %g.", tmp,
+ *((float *)(p->std->array)) )<0 )
+ error(EXIT_FAILURE, 0, "%s: asprintf allocation", __func__);
+ }
+ else
+ {
+ if( asprintf(&str, "Sky %s: %s (hdu: %s).", tmp, p->usedstdfile,
+ p->stdhdu)<0 )
+ error(EXIT_FAILURE, 0, "%s: asprintf allocation", __func__);
+ }
+ gal_list_str_add(comments, str, 0);
+ }
+
+ /* Upper limit mask. */
+ if(p->upmaskfile)
+ {
+ if( asprintf(&str, "Upperlimit mask: %s (hdu: %s).", p->upmaskfile,
+ p->upmaskhdu)<0 )
+ error(EXIT_FAILURE, 0, "%s: asprintf allocation", __func__);
+ gal_list_str_add(comments, str, 0);
+ }
+}
+
+
+
+
+
/* Write the similar information. */
static gal_list_str_t *
mkcatalog_outputs_same_start(struct mkcatalogparams *p, int o0c1,
char *ObjClump)
{
- float snlim;
char *str, *tstr;
double pixarea=NAN;
gal_list_str_t *comments=NULL;
- char *skyfile=p->skyfile ? p->skyfile : p->inputname;
- char *stdfile=p->stdfile ? p->stdfile : p->inputname;
- char *clumpsfile=p->clumpsfile ? p->clumpsfile : p->inputname;
- char *objectsfile=p->objectsfile ? p->objectsfile : p->inputname;
if( asprintf(&str, "%s catalog of %s", o0c1 ? "Object" : "Clump",
PROGRAM_STRING)<0 )
@@ -824,44 +429,11 @@ mkcatalog_outputs_same_start(struct mkcatalogparams *p,
int o0c1,
gal_list_str_add(&comments, str, 0);
- if(p->cp.tableformat==GAL_TABLE_FORMAT_TXT)
- {
- if( asprintf(&str, "--------- Input files ---------")<0 )
- error(EXIT_FAILURE, 0, "%s: asprintf allocation", __func__);
- gal_list_str_add(&comments, str, 0);
- }
-
- if( asprintf(&str, "Values: %s (hdu: %s).", p->inputname, p->cp.hdu)<0 )
- error(EXIT_FAILURE, 0, "%s: asprintf allocation", __func__);
- gal_list_str_add(&comments, str, 0);
-
- if( asprintf(&str, "Objects: %s (hdu: %s).", objectsfile, p->objectshdu)<0 )
- error(EXIT_FAILURE, 0, "%s: asprintf allocation", __func__);
- gal_list_str_add(&comments, str, 0);
+ /* Write the basic information. */
+ mkcatalog_write_inputs_in_comments(p, &comments, 1, 1);
- if(p->clumps)
- {
- if(asprintf(&str, "Clumps: %s (hdu: %s).", clumpsfile, p->clumpshdu)<0)
- error(EXIT_FAILURE, 0, "%s: asprintf allocation", __func__);
- gal_list_str_add(&comments, str, 0);
- }
-
- if( asprintf(&str, "Sky: %s (hdu: %s).", skyfile, p->skyhdu)<0 )
- error(EXIT_FAILURE, 0, "%s: asprintf allocation", __func__);
- gal_list_str_add(&comments, str, 0);
-
- if( asprintf(&str, "Sky STD: %s (hdu: %s).", stdfile, p->stdhdu)<0 )
- error(EXIT_FAILURE, 0, "%s: asprintf allocation", __func__);
- gal_list_str_add(&comments, str, 0);
-
- if(p->upmaskfile)
- {
- if( asprintf(&str, "Upperlimit mask: %s (hdu: %s).", p->upmaskfile,
- p->upmaskhdu)<0 )
- error(EXIT_FAILURE, 0, "%s: asprintf allocation", __func__);
- gal_list_str_add(&comments, str, 0);
- }
+ /* Write other supplimentary information. */
if(p->cp.tableformat==GAL_TABLE_FORMAT_TXT)
{
if( asprintf(&str, "--------- Supplimentary information ---------")<0 )
@@ -869,9 +441,9 @@ mkcatalog_outputs_same_start(struct mkcatalogparams *p, int
o0c1,
gal_list_str_add(&comments, str, 0);
}
- if(p->input->wcs)
+ if(p->objects->wcs)
{
- pixarea=gal_wcs_pixel_area_arcsec2(p->input->wcs);
+ pixarea=gal_wcs_pixel_area_arcsec2(p->objects->wcs);
if( isnan(pixarea)==0 )
{
if( asprintf(&str, "Pixel area (arcsec^2): %g", pixarea)<0 )
@@ -888,7 +460,7 @@ mkcatalog_outputs_same_start(struct mkcatalogparams *p, int
o0c1,
}
/* Print surface brightness limits. */
- if( !isnan(p->zeropoint) && !isnan(p->sfmagnsigma) )
+ if( !isnan(p->medstd) && !isnan(p->zeropoint) && !isnan(p->sfmagnsigma) )
{
/* Per pixel. */
if( asprintf(&str, "%g sigma surface brightness (magnitude/pixel): "
@@ -942,23 +514,6 @@ mkcatalog_outputs_same_start(struct mkcatalogparams *p,
int o0c1,
gal_list_str_add(&comments, str, 0);
}
- snlim = o0c1 ? p->clumpsn : p->detsn;
- if( !isnan(snlim) )
- {
- if( asprintf(&str, "%s limiting signal-to-noise ratio: %.3f", ObjClump,
- snlim)<0 )
- error(EXIT_FAILURE, 0, "%s: asprintf allocation", __func__);
- gal_list_str_add(&comments, str, 0);
- }
-
- if(o0c1==0)
- {
- if( asprintf(&str, "(NOTE: S/N limit above is for pseudo-detections, "
- "not objects.)")<0 )
- error(EXIT_FAILURE, 0, "%s: asprintf allocation", __func__);
- gal_list_str_add(&comments, str, 0);
- }
-
if(p->cpscorr>1.0f)
{
if( asprintf(&str, "Counts-per-second correction: %.3f", p->cpscorr)<0 )
@@ -966,96 +521,75 @@ mkcatalog_outputs_same_start(struct mkcatalogparams *p,
int o0c1,
gal_list_str_add(&comments, str, 0);
}
- if( !isnan(p->threshold) )
- {
- if( asprintf(&str, "**IMPORTANT** Pixel threshold (multiple of local "
- "std): %.3f", p->threshold)<0 )
- error(EXIT_FAILURE, 0, "%s: asprintf allocation", __func__);
- gal_list_str_add(&comments, str, 0);
- }
-
-
if(p->upperlimit)
- {
- if(p->cp.tableformat==GAL_TABLE_FORMAT_TXT)
- {
- if(asprintf(&str, "--------- Upper-limit measurement ---------")<0)
- error(EXIT_FAILURE, 0, "%s: asprintf allocation", __func__);
- gal_list_str_add(&comments, str, 0);
- }
+ upperlimit_write_comments(p, &comments, 1);
- if( asprintf(&str, "Number of random samples: %zu", p->upnum)<0 )
- error(EXIT_FAILURE, 0, "%s: asprintf allocation", __func__);
- gal_list_str_add(&comments, str, 0);
- if(p->uprange)
- {
- switch(p->input->ndim)
- {
- case 2:
- if( asprintf(&str, "Range of random samples about target: "
- "%zu, %zu", p->uprange[1], p->uprange[0])<0 )
- error(EXIT_FAILURE, 0, "%s: asprintf allocation", __func__);
- break;
- case 3:
- if( asprintf(&str, "Range of random samples about target: %zu, "
- "%zu, %zu", p->uprange[2], p->uprange[1],
- p->uprange[0])<0 )
- error(EXIT_FAILURE, 0, "%s: asprintf allocation", __func__);
- break;
- default:
- error(EXIT_FAILURE, 0, "%s: a bug! Please contact us at %s to "
- "address the problem. The value %zu is not recognized for "
- "`p->input->ndim'", __func__, PACKAGE_BUGREPORT,
- p->input->ndim);
- }
- gal_list_str_add(&comments, str, 0);
- }
- if( asprintf(&str, "Random number generator name: %s", p->rngname)<0 )
+ if(p->cp.tableformat==GAL_TABLE_FORMAT_TXT)
+ {
+ if( asprintf(&str, "--------- Table columns ---------")<0 )
error(EXIT_FAILURE, 0, "%s: asprintf allocation", __func__);
gal_list_str_add(&comments, str, 0);
+ }
- if( asprintf(&str, "Random number generator seed: %"PRIu64, p->seed)<0 )
- error(EXIT_FAILURE, 0, "%s: asprintf allocation", __func__);
- gal_list_str_add(&comments, str, 0);
+ /* Return the comments. */
+ return comments;
+}
- if( asprintf(&str, "Multiple of STD used for sigma-clipping: %.3f",
- p->upsigmaclip[0])<0 )
- error(EXIT_FAILURE, 0, "%s: asprintf allocation", __func__);
- gal_list_str_add(&comments, str, 0);
- if(p->upsigmaclip[1]>=1.0f)
- {
- if( asprintf(&str, "Number of clips for sigma-clipping: %.0f",
- p->upsigmaclip[1])<0 )
- error(EXIT_FAILURE, 0, "%s: asprintf allocation", __func__);
- }
- else
- {
- if( asprintf(&str, "Tolerance level to sigma-clipping: %.3f",
- p->upsigmaclip[1])<0 )
- error(EXIT_FAILURE, 0, "%s: asprintf allocation", __func__);
- }
- gal_list_str_add(&comments, str, 0);
- if( asprintf(&str, "Multiple of sigma-clipped STD for upper-limit: "
- "%.3f", p->upnsigma)<0 )
- error(EXIT_FAILURE, 0, "%s: asprintf allocation", __func__);
- gal_list_str_add(&comments, str, 0);
- }
- if(p->cp.tableformat==GAL_TABLE_FORMAT_TXT)
+/* Since all the measurements were done in parallel (and we didn't know the
+ number of clumps per object a-priori), the clumps informtion is just
+ written in as they are measured. Here, we'll sort the clump columns by
+ object ID. There is an option to disable this. */
+static void
+sort_clumps_by_objid(struct mkcatalogparams *p)
+{
+ gal_data_t *col;
+ size_t o, i, j, *permute, *rowstart;
+
+ /* Make sure everything is fine. */
+ if(p->hostobjid_c==NULL || p->numclumps_c==NULL)
+ error(EXIT_FAILURE, 0, "%s: a bug! Please contact us at %s to fix the "
+ "problem. `p->hostobjid_c' and `p->numclumps_c' must not be "
+ "NULL.", __func__, PACKAGE_BUGREPORT);
+
+
+ /* Allocate the necessary arrays. */
+ rowstart=gal_data_malloc_array(GAL_TYPE_SIZE_T, p->numobjects, __func__,
+ "rowstart");
+ permute=gal_data_malloc_array(GAL_TYPE_SIZE_T, p->numclumps, __func__,
+ "permute");
+
+
+ /* The objects array is already sorted by object ID. So we should just
+ add up the number of clumps to find the row where each object's clumps
+ should start from in the final sorted clumps catalog. */
+ rowstart[0] = 0;
+ for(i=1;i<p->numobjects;++i)
+ rowstart[i] = p->numclumps_c[i-1] + rowstart[i-1];
+
+ /* Fill the permutation array. Note that WE KNOW that all the objects for
+ one clump are after each other.*/
+ i=0;
+ while(i<p->numclumps)
{
- if( asprintf(&str, "--------- Table columns ---------")<0 )
- error(EXIT_FAILURE, 0, "%s: asprintf allocation", __func__);
- gal_list_str_add(&comments, str, 0);
+ o=p->hostobjid_c[i]-1;
+ for(j=0; j<p->numclumps_c[o]; ++j)
+ permute[i++] = rowstart[o] + j;
}
- /* Return the comments. */
- return comments;
+ /* Permute all the clump columns. */
+ for(col=p->clumpcols; col!=NULL; col=col->next)
+ gal_permutation_apply_inverse(col, permute);
+
+ /* Clean up */
+ free(permute);
+ free(rowstart);
}
@@ -1091,10 +625,9 @@ mkcatalog_write_outputs(struct mkcatalogparams *p)
============== */
if(p->clumps)
{
+ /* Make the comments. */
comments=mkcatalog_outputs_same_start(p, 1, "Clumps");
-
-
/* Write objects catalog
---------------------
@@ -1105,6 +638,19 @@ mkcatalog_write_outputs(struct mkcatalogparams *p)
"CLUMPS");
gal_list_str_free(comments, 1);
}
+
+ /* Inform the user. */
+ if(!p->cp.quiet)
+ {
+ if(p->clumpsout==p->objectsout)
+ printf(" - Output catalog: %s\n", p->objectsout);
+ else
+ {
+ printf(" - Output objects catalog: %s\n", p->objectsout);
+ if(p->clumps)
+ printf(" - Output clumps catalog: %s\n", p->clumpsout);
+ }
+ }
}
@@ -1136,21 +682,22 @@ mkcatalog(struct mkcatalogparams *p)
it to assign a column to the clumps in the final catalog. */
if( p->cp.numthreads > 1 ) pthread_mutex_init(&p->mutex, NULL);
-
/* Do the processing on each thread. */
gal_threads_spin_off(mkcatalog_single_object, p, p->numobjects,
p->cp.numthreads);
-
/* Post-thread processing, for example to convert image coordinates to RA
and Dec. */
mkcatalog_wcs_conversion(p);
+ /* If the columns need to be sorted (by object ID), then some adjustments
+ need to be made (possibly to both the objects and clumps catalogs). */
+ if(p->hostobjid_c)
+ sort_clumps_by_objid(p);
/* Write the filled columns into the output. */
mkcatalog_write_outputs(p);
-
/* Destroy the mutex. */
if( p->cp.numthreads>1 ) pthread_mutex_destroy(&p->mutex);
}
diff --git a/bin/mkcatalog/mkcatalog.h b/bin/mkcatalog/mkcatalog.h
index 4e93c35..be94551 100644
--- a/bin/mkcatalog/mkcatalog.h
+++ b/bin/mkcatalog/mkcatalog.h
@@ -31,11 +31,11 @@ struct mkcatalog_passparams
int32_t object; /* Object that is currently working on. */
size_t clumpsinobj; /* The number of clumps in this object. */
gal_data_t *tile; /* The tile to pass-over. */
- float *st_i; /* Starting pointer for input image. */
- int32_t *st_o; /* Starting pointer for objects image. */
- int32_t *st_c; /* Starting pointer for clumps image. */
- float *st_sky; /* Starting pointer for input image. */
- float *st_std; /* Starting pointer for input image. */
+ int32_t *st_o; /* Starting pointer for object labels. */
+ int32_t *st_c; /* Starting pointer for clump labels. */
+ float *st_v; /* Starting pointer for values array. */
+ float *st_sky; /* Starting pointer for Sky array. */
+ float *st_std; /* Starting pointer for Sky STD array. */
size_t start_end_inc[2]; /* Starting and ending indexs. */
size_t *shift; /* Shift coordinates. */
gsl_rng *rng; /* Random number generator. */
@@ -44,6 +44,11 @@ struct mkcatalog_passparams
};
void
+mkcatalog_write_inputs_in_comments(struct mkcatalogparams *p,
+ gal_list_str_t **comments, int withsky,
+ int withstd);
+
+void
mkcatalog(struct mkcatalogparams *p);
#endif
diff --git a/bin/mkcatalog/parse.c b/bin/mkcatalog/parse.c
new file mode 100644
index 0000000..c5a22e2
--- /dev/null
+++ b/bin/mkcatalog/parse.c
@@ -0,0 +1,670 @@
+/*********************************************************************
+MakeCatalog - Make a catalog from an input and labeled image.
+MakeCatalog is part of GNU Astronomy Utilities (Gnuastro) package.
+
+Original author:
+ Mohammad Akhlaghi <mohammad@akhlaghi.org>
+Contributing author(s):
+Copyright (C) 2018, Free Software Foundation, Inc.
+
+Gnuastro is free software: you can redistribute it and/or modify it
+under the terms of the GNU General Public License as published by the
+Free Software Foundation, either version 3 of the License, or (at your
+option) any later version.
+
+Gnuastro is distributed in the hope that it will be useful, but
+WITHOUT ANY WARRANTY; without even the implied warranty of
+MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the GNU
+General Public License for more details.
+
+You should have received a copy of the GNU General Public License
+along with Gnuastro. If not, see <http://www.gnu.org/licenses/>.
+**********************************************************************/
+#include <config.h>
+
+#include <math.h>
+#include <stdio.h>
+#include <errno.h>
+#include <error.h>
+#include <float.h>
+#include <string.h>
+#include <stdlib.h>
+
+#include <gnuastro/data.h>
+#include <gnuastro/dimension.h>
+#include <gnuastro/statistics.h>
+
+#include "main.h"
+#include "mkcatalog.h"
+
+#include "parse.h"
+
+
+
+
+
+/* Both passes are going to need their starting pointers set, so we'll do
+ that here. */
+void
+parse_initialize(struct mkcatalog_passparams *pp)
+{
+ struct mkcatalogparams *p=pp->p;
+
+ size_t i, ndim=p->objects->ndim;
+ size_t *start_end=pp->start_end_inc;
+
+ /* Initialize the number of clumps in this object. */
+ pp->clumpsinobj=0;
+
+
+ /* Initialize the intermediate values to zero. */
+ memset(pp->oi, 0, OCOL_NUMCOLS * sizeof *pp->oi);
+
+
+ /* Set the shifts in every dimension to avoid round-off errors in large
+ numbers for the non-linear calculations. We are using the first pixel
+ of each object's tile as the shift parameter to keep the mean
+ (average) reasonably near to the standard deviation. Otherwise, when
+ the object is far out in the image (large x and y positions), then
+ roundoff errors are going to decrease the accuracy of the second order
+ calculations. */
+ if(pp->shift)
+ {
+ /* Get the coordinates of the tile's starting point. */
+ gal_dimension_index_to_coord( ( (float *)(pp->tile->array)
+ - (float *)(pp->tile->block->array) ),
+ ndim, p->objects->dsize, pp->shift);
+
+ /* Change their counting to start from 1, not zero, since we will be
+ using them as FITS coordinates. */
+ for(i=0;i<ndim;++i) ++pp->shift[i];
+ }
+
+
+ /* Set the starting and ending indexs of this tile/object on all (the
+ possible) input arrays. */
+ pp->st_o = gal_tile_start_end_ind_inclusive(pp->tile, p->objects,
+ start_end);
+ pp->st_c = (p->clumps
+ ? (int32_t *)(p->clumps->array) + start_end[0] : NULL);
+ pp->st_v = (p->values
+ ? (float *)(p->values->array) + start_end[0] : NULL);
+ pp->st_sky = ( p->sky
+ ? ( p->sky->size==p->objects->size
+ ? (float *)(p->sky->array) + start_end[0]
+ : NULL )
+ : NULL);
+ pp->st_std = ( p->std
+ ? ( p->std->size==p->objects->size
+ ? (float *)(p->std->array) + start_end[0]
+ : NULL )
+ : NULL );
+}
+
+
+
+
+
+void
+parse_objects(struct mkcatalog_passparams *pp)
+{
+ uint8_t *oif=pp->p->oiflag;
+ struct mkcatalogparams *p=pp->p;
+ size_t ndim=p->objects->ndim, *dsize=p->objects->dsize;
+
+ double *oi=pp->oi;
+ size_t d, increment=0, num_increment=1;
+ float st, sval, *V=NULL, *SK=NULL, *ST=NULL;
+ int32_t *O, *OO, *C=NULL, *objects=p->objects->array;
+ float *std=p->std?p->std->array:NULL, *sky=p->sky?p->sky->array:NULL;
+
+ /* If tile processing isn't necessary, set `tid' to a blank value. */
+ size_t tid = ( ( (p->sky && p->sky->size>1 && pp->st_sky == NULL )
+ || ( p->std && p->std->size>1 && pp->st_std == NULL ) )
+ ? 0 : GAL_BLANK_SIZE_T );
+
+ /* Coordinate shift. */
+ size_t *sc = ( pp->shift
+ ? gal_data_malloc_array(GAL_TYPE_SIZE_T, ndim, __func__,
+ "sc")
+ : NULL );
+
+ /* If any coordinate columns are requested. */
+ size_t *c = (
+ /* Coordinate-related columns. */
+ ( oif[ OCOL_GX ]
+ || oif[ OCOL_GY ]
+ || oif[ OCOL_GZ ]
+ || oif[ OCOL_VX ]
+ || oif[ OCOL_VY ]
+ || oif[ OCOL_VZ ]
+ || oif[ OCOL_C_GX ]
+ || oif[ OCOL_C_GY ]
+ || oif[ OCOL_C_GZ ]
+ || sc
+ /* When the sky and its STD are tiles, we'll also need
+ the coordinate to find which tile a pixel belongs
+ to. */
+ || tid==GAL_BLANK_SIZE_T )
+ ? gal_data_malloc_array(GAL_TYPE_SIZE_T, ndim, __func__, "c")
+ : NULL );
+
+
+ /* Parse each contiguous patch of memory covered by this object. */
+ while( pp->start_end_inc[0] + increment <= pp->start_end_inc[1] )
+ {
+ /* Set the contiguous range to parse. The pixel-to-pixel counting
+ along the fastest dimension will be done over the `O' pointer. */
+ if( p->clumps ) C = pp->st_c + increment;
+ if( p->values ) V = pp->st_v + increment;
+ if( p->sky && pp->st_sky ) SK = pp->st_sky + increment;
+ if( p->std && pp->st_std ) ST = pp->st_std + increment;
+ OO = ( O = pp->st_o + increment ) + pp->tile->dsize[ndim-1];
+
+ /* Parse the tile. */
+ do
+ {
+ /* If this pixel belongs to the requested object then do the
+ processing. */
+ if( *O==pp->object )
+ {
+ /* INTERNAL: Get the number of clumps in this object: it is
+ the largest clump ID over each object. */
+ if( p->clumps && *C>0 )
+ pp->clumpsinobj = *C > pp->clumpsinobj ? *C : pp->clumpsinobj;
+
+
+ /* Add to the area of this object. */
+ if(oif[ OCOL_NUMALL ]) oi[ OCOL_NUMALL ]++;
+
+
+ /* Geometric coordinate measurements. */
+ if(c)
+ {
+ /* Convert the index to coordinate. */
+ gal_dimension_index_to_coord(O-objects, ndim, dsize, c);
+
+ /* If we need tile-ID, get the tile ID now. */
+ if(tid!=GAL_BLANK_SIZE_T)
+ tid=gal_tile_full_id_from_coord(&p->cp.tl, c);
+
+ /* Do the general geometric (independent of pixel value)
+ calculations. */
+ if(oif[ OCOL_GX ]) oi[ OCOL_GX ] += c[ ndim-1 ]+1;
+ if(oif[ OCOL_GY ]) oi[ OCOL_GY ] += c[ ndim-2 ]+1;
+ if(oif[ OCOL_GZ ]) oi[ OCOL_GZ ] += c[ ndim-3 ]+1;
+ if(pp->shift)
+ {
+ /* Calculate the shifted coordinates for second order
+ calculations. The coordinate is incremented because
+ from now on, the positions are in the FITS standard
+ (starting from one). */
+ for(d=0;d<ndim;++d) sc[d] = c[d] + 1 - pp->shift[d];
+
+ /* Include the shifted values, note that the second
+ order moments are never needed independently, they
+ are used together to find the ellipticity
+ parameters. */
+ oi[ OCOL_GXX ] += sc[1] * sc[1];
+ oi[ OCOL_GYY ] += sc[0] * sc[0];
+ oi[ OCOL_GXY ] += sc[1] * sc[0];
+ }
+ if(p->clumps && *C>0)
+ {
+ if(oif[ OCOL_C_NUMALL ]) oi[ OCOL_C_NUMALL ]++;
+ if(oif[ OCOL_C_GX ]) oi[ OCOL_C_GX ] += c[ ndim-1 ]+1;
+ if(oif[ OCOL_C_GY ]) oi[ OCOL_C_GY ] += c[ ndim-2 ]+1;
+ if(oif[ OCOL_C_GZ ]) oi[ OCOL_C_GZ ] += c[ ndim-3 ]+1;
+ }
+ }
+
+
+ /* Value related measurements. */
+ if( p->values && !( p->hasblank && isnan(*V) ) )
+ {
+ /* General flux summations. */
+ if(oif[ OCOL_NUM ]) oi[ OCOL_NUM ]++;
+ if(oif[ OCOL_SUM ]) oi[ OCOL_SUM ] += *V;
+
+ /* Get the necessary clump information. */
+ if(p->clumps && *C>0)
+ {
+ if(oif[ OCOL_C_NUM ]) oi[ OCOL_C_NUM ]++;
+ if(oif[ OCOL_C_SUM ]) oi[ OCOL_C_SUM ] += *V;
+ }
+
+ /* For flux weighted centers, we can only use positive
+ values, so do those measurements here. */
+ if( *V > 0.0f )
+ {
+ if(oif[ OCOL_NUMWHT ]) oi[ OCOL_NUMWHT ]++;
+ if(oif[ OCOL_SUMWHT ]) oi[ OCOL_SUMWHT ] += *V;
+ if(oif[ OCOL_VX ]) oi[ OCOL_VX ] += *V*(c[ ndim-1 ]+1);
+ if(oif[ OCOL_VY ]) oi[ OCOL_VY ] += *V*(c[ ndim-2 ]+1);
+ if(oif[ OCOL_VZ ]) oi[ OCOL_VZ ] += *V*(c[ ndim-3 ]+1);
+ if(pp->shift)
+ {
+ oi[ OCOL_VXX ] += *V * sc[1] * sc[1];
+ oi[ OCOL_VYY ] += *V * sc[0] * sc[0];
+ oi[ OCOL_VXY ] += *V * sc[1] * sc[0];
+ }
+ if(p->clumps && *C>0)
+ {
+ if(oif[ OCOL_C_NUMWHT ]) oi[ OCOL_C_NUMWHT ]++;
+ if(oif[ OCOL_C_SUMWHT ]) oi[ OCOL_C_SUMWHT ] += *V;
+ if(oif[ OCOL_C_VX ])
+ oi[ OCOL_C_VX ] += *V * (c[ ndim-1 ]+1);
+ if(oif[ OCOL_C_VY ])
+ oi[ OCOL_C_VY ] += *V * (c[ ndim-2 ]+1);
+ if(oif[ OCOL_C_VZ ])
+ oi[ OCOL_C_VZ ] += *V * (c[ ndim-3 ]+1);
+ }
+ }
+ }
+
+
+ /* Sky value based measurements. */
+ if(p->sky)
+ if(oif[ OCOL_SUMSKY ])
+ oi[ OCOL_SUMSKY ] += ( pp->st_sky
+ ? *SK /* Full array */
+ : ( p->sky->size>1
+ ? sky[tid] /* Tile */
+ : sky[0] ) ); /* Single value */
+
+
+ /* Sky standard deviation based measurements.*/
+ if(p->std)
+ {
+ sval = pp->st_std ? *ST : (p->std->size>1?std[tid]:std[0]);
+ st = p->variance ? sqrt(sval) : sval;
+ if(oif[ OCOL_SUMSTD ]) oi[ OCOL_SUMSTD ] += st;
+ /* For each pixel, we have a sky contribution to the
+ counts and the signal's contribution. The standard
+ deviation in the sky is simply `sval', but the
+ standard deviation of the signal (independent of the
+ sky) is `sqrt(*V)'. Therefore the total variance of
+ this pixel is the variance of the sky added with the
+ absolute value of its sky-subtracted flux. We use the
+ absolute value, because especially as the signal gets
+ noisy there will be negative values, and we don't want
+ them to decrease the variance. */
+ if(oif[ OCOL_SUM_VAR ])
+ oi[ OCOL_SUM_VAR ] += ( (p->variance ? sval : st*st)
+ + fabs(*V) );
+ }
+ }
+
+ /* Increment the other pointers. */
+ if( p->values ) ++V;
+ if( p->clumps ) ++C;
+ if( p->sky && pp->st_sky ) ++SK;
+ if( p->std && pp->st_std ) ++ST;
+ }
+ while(++O<OO);
+
+ /* Increment to the next contiguous region of this tile. */
+ increment += ( gal_tile_block_increment(p->objects, dsize,
+ num_increment++, NULL) );
+ }
+
+ /* Clean up. */
+ if(c) free(c);
+ if(sc) free(sc);
+}
+
+
+
+
+
+/* Parse over the clumps within an object. */
+void
+parse_clumps(struct mkcatalog_passparams *pp)
+{
+ struct mkcatalogparams *p=pp->p;
+ size_t ndim=p->objects->ndim, *dsize=p->objects->dsize;
+
+ double *ci, *cir;
+ uint8_t *cif=p->ciflag;
+ int32_t *O, *OO, *C=NULL, nlab;
+ float st, sval, *V=NULL, *SK=NULL, *ST=NULL;
+ size_t i, ii, d, increment=0, num_increment=1;
+ size_t nngb=gal_dimension_num_neighbors(ndim);
+ int32_t *objects=p->objects->array, *clumps=p->clumps->array;
+ float *std=p->std?p->std->array:NULL, *sky=p->sky?p->sky->array:NULL;
+
+ /* If tile processing isn't necessary, set `tid' to a blank value. */
+ size_t tid = ( ( (p->sky && p->sky->size>1 && pp->st_sky == NULL )
+ || ( p->std && p->std->size>1 && pp->st_std == NULL ) )
+ ? 0 : GAL_BLANK_SIZE_T );
+
+ /* Coordinate shift. */
+ size_t *sc = ( pp->shift
+ ? gal_data_malloc_array(GAL_TYPE_SIZE_T, ndim, __func__,
+ "sc")
+ : NULL );
+
+ /* If any coordinate columns are requested. */
+ size_t *c = ( ( cif[ CCOL_GX ]
+ || cif[ CCOL_GY ]
+ || cif[ CCOL_GZ ]
+ || cif[ CCOL_VX ]
+ || cif[ CCOL_VY ]
+ || cif[ CCOL_VZ ]
+ || sc
+ || tid==GAL_BLANK_SIZE_T )
+ ? gal_data_malloc_array(GAL_TYPE_SIZE_T, ndim, __func__, "c")
+ : NULL );
+
+ /* Preparations for neighbor parsing. */
+ int32_t *ngblabs=( ( cif[ CCOL_RIV_NUM ]
+ || cif[ CCOL_RIV_SUM ]
+ || cif[ CCOL_RIV_SUM_VAR ] )
+ ? gal_data_malloc_array(GAL_TYPE_INT32, nngb, __func__,
+ "ngblabs")
+ : NULL );
+ size_t *dinc = ngblabs ? gal_dimension_increment(ndim, dsize) : NULL;
+
+
+ /* Parse each contiguous patch of memory covered by this object. */
+ while( pp->start_end_inc[0] + increment <= pp->start_end_inc[1] )
+ {
+ /* Set the contiguous range to parse. The pixel-to-pixel counting
+ along the fastest dimension will be done over the `O' pointer. */
+ C = pp->st_c + increment;
+ if( p->values ) V = pp->st_v + increment;
+ if( p->sky && pp->st_sky ) SK = pp->st_sky + increment;
+ if( p->std && pp->st_std ) ST = pp->st_std + increment;
+ OO = ( O = pp->st_o + increment ) + pp->tile->dsize[ndim-1];
+
+ /* Parse the tile */
+ do
+ {
+ /* If this pixel belongs to the requested object then do the
+ processing. */
+ if( *O==pp->object )
+ {
+ /* We are on a clump. */
+ if(p->clumps && *C>0)
+ {
+ /* Pointer to make things easier. Note that the clump
+ labels start from 1, but the array indexs from 0.*/
+ ci=&pp->ci[ (*C-1) * CCOL_NUMCOLS ];
+
+ /* Add to the area of this object. */
+ if(cif[ CCOL_NUMALL ]) ci[ CCOL_NUMALL ]++;
+
+ /* Raw-position related measurements. */
+ if(c)
+ {
+ /* Get "C" the coordinates of this point. */
+ gal_dimension_index_to_coord(O-objects, ndim, dsize, c);
+
+ /* If we need tile-ID, get the tile ID now. */
+ if(tid!=GAL_BLANK_SIZE_T)
+ tid=gal_tile_full_id_from_coord(&p->cp.tl, c);
+
+ /* General geometric (independent of pixel value)
+ calculations. */
+ if(cif[ CCOL_GX ]) ci[ CCOL_GX ] += c[ ndim-1 ]+1;
+ if(cif[ CCOL_GY ]) ci[ CCOL_GY ] += c[ ndim-2 ]+1;
+ if(cif[ CCOL_GZ ]) ci[ CCOL_GZ ] += c[ ndim-3 ]+1;
+ if(pp->shift)
+ {
+ /* Shifted coordinates for second order moments,
+ see explanations in the first pass.*/
+ for(d=0;d<ndim;++d) sc[d] = c[d] + 1 - pp->shift[d];
+
+ /* Raw second-order measurements. */
+ ci[ CCOL_GXX ] += sc[1] * sc[1];
+ ci[ CCOL_GYY ] += sc[0] * sc[0];
+ ci[ CCOL_GXY ] += sc[1] * sc[0];
+ }
+ }
+
+ /* Value related measurements, see `parse_objects' for
+ comments. */
+ if( p->values && !( p->hasblank && isnan(*V) ) )
+ {
+ /* Fill in the necessary information. */
+ if(cif[ CCOL_NUM ]) ci[ CCOL_NUM ]++;
+ if(cif[ CCOL_SUM ]) ci[ CCOL_SUM ] += *V;
+ if( *V > 0.0f )
+ {
+ if(cif[ CCOL_NUMWHT ]) ci[ CCOL_NUMWHT ]++;
+ if(cif[ CCOL_SUMWHT ]) ci[ CCOL_SUMWHT ] += *V;
+ if(cif[ CCOL_VX ])
+ ci[ CCOL_VX ] += *V * (c[ ndim-1 ]+1);
+ if(cif[ CCOL_VY ])
+ ci[ CCOL_VY ] += *V * (c[ ndim-2 ]+1);
+ if(cif[ CCOL_VZ ])
+ ci[ CCOL_VZ ] += *V * (c[ ndim-3 ]+1);
+ if(pp->shift)
+ {
+ ci[ CCOL_VXX ] += *V * sc[1] * sc[1];
+ ci[ CCOL_VYY ] += *V * sc[0] * sc[0];
+ ci[ CCOL_VXY ] += *V * sc[1] * sc[0];
+ }
+ }
+ }
+
+ /* Sky based measurements. */
+ if(p->sky)
+ if(cif[ CCOL_SUMSKY ])
+ ci[ CCOL_SUMSKY ] += ( pp->st_sky
+ ? *SK /* Full */
+ : ( p->sky->size>1
+ ? sky[tid] /* Tile */
+ : sky[0] ) ); /* 1 value */
+
+ /* Sky Standard deviation based measurements, see
+ `parse_objects' for comments. */
+ if(p->std)
+ {
+ sval = ( pp->st_std
+ ? *ST
+ : (p->std->size>1 ? std[tid] : std[0]) );
+ st = p->variance ? sqrt(sval) : sval;
+ if(cif[ CCOL_SUMSTD ]) ci[ CCOL_SUMSTD ] += st;
+ if(cif[ CCOL_SUM_VAR ])
+ ci[ CCOL_SUM_VAR ] += ( (p->variance ? sval : st*st)
+ + fabs(*V) );
+ }
+ }
+
+ /* This pixel is on the diffuse region (and the object
+ actually has clumps). If any river-based measurements are
+ necessary check to see if it is touching a clump or not,
+ but only if this object actually has any clumps. */
+ else if(ngblabs && pp->clumpsinobj)
+ {
+ /* We are on a diffuse (possibly a river) pixel. So the
+ value of this pixel has to be added to any of the
+ clumps in touches. But since it might touch a labeled
+ region more than once, we use `ngblabs' to keep track
+ of which label we have already added its value
+ to. `ii' is the number of different labels this river
+ pixel has already been considered for. `ngblabs' will
+ keep the list labels. */
+ ii=0;
+ memset(ngblabs, 0, nngb*sizeof *ngblabs);
+
+ /* Go over the neighbors and see if this pixel is
+ touching a clump or not. */
+ GAL_DIMENSION_NEIGHBOR_OP(O-objects, ndim, dsize, ndim,
+ dinc,
+ {
+ /* Neighbor's label (mainly for easy reading). */
+ nlab=clumps[nind];
+
+ /* We only want neighbors that are a clump and part
+ of this object and part of the same object. */
+ if( nlab>0 && objects[nind]==pp->object)
+ {
+ /* Go over all already checked labels and make
+ sure this clump hasn't already been
+ considered. */
+ for(i=0;i<ii;++i) if(ngblabs[i]==nlab) break;
+
+ /* It hasn't been considered yet: */
+ if(i==ii)
+ {
+ /* Make sure it won't be considered any
+ more. */
+ ngblabs[ii++] = nlab;
+
+ /* To help in reading. */
+ cir=&pp->ci[ (nlab-1) * CCOL_NUMCOLS ];
+
+ /* Write in the necessary values. */
+ if(cif[ CCOL_RIV_NUM ])
+ cir[ CCOL_RIV_NUM ]++;
+
+ if(cif[ CCOL_RIV_SUM ])
+ cir[ CCOL_RIV_SUM ] += *V;
+
+ if(cif[ CCOL_RIV_SUM_VAR ])
+ {
+ sval = ( pp->st_std
+ ? *ST
+ : ( p->std->size>1
+ ? std[tid]
+ : std[0] ) );
+ cir[ CCOL_RIV_SUM_VAR ] += fabs(*V)
+ + (p->variance ? sval : sval*sval);
+ }
+ }
+ }
+ });
+ }
+ }
+
+ /* Increment the other pointers. */
+ ++C;
+ if( p->values ) ++V;
+ if( p->sky && pp->st_sky ) ++SK;
+ if( p->std && pp->st_std ) ++ST;
+ }
+ while(++O<OO);
+
+ /* Increment to the next contiguous region of this tile. */
+ increment += ( gal_tile_block_increment(p->objects, dsize,
+ num_increment++, NULL) );
+ }
+
+ /* Clean up. */
+ if(c) free(c);
+ if(sc) free(sc);
+ if(dinc) free(dinc);
+ if(ngblabs) free(ngblabs);
+}
+
+
+
+
+
+void
+parse_median(struct mkcatalog_passparams *pp)
+{
+ struct mkcatalogparams *p=pp->p;
+ size_t ndim=p->objects->ndim, *dsize=p->objects->dsize;
+
+ float *V;
+ double *ci;
+ gal_data_t *median;
+ int32_t *O, *OO, *C=NULL;
+ gal_data_t **clumpsmed=NULL;
+ size_t i, increment=0, num_increment=1;
+ size_t counter=0, *ccounter=NULL, tsize=pp->oi[OCOL_NUM];
+ gal_data_t *objmed=gal_data_alloc(NULL, p->values->type, 1, &tsize, NULL, 0,
+ p->cp.minmapsize, NULL, NULL, NULL);
+
+ /* Allocate space for the clump medians. */
+ if(p->clumps)
+ {
+ errno=0;
+ clumpsmed=malloc(pp->clumpsinobj * sizeof *clumpsmed);
+ if(clumpsmed==NULL)
+ error(EXIT_FAILURE, errno, "%s: couldn't allocate `clumpsmed' for "
+ "%zu clumps", __func__, pp->clumpsinobj);
+
+
+ /* Allocate the array necessary to keep the values of each clump. */
+ ccounter=gal_data_calloc_array(GAL_TYPE_SIZE_T, pp->clumpsinobj,
+ __func__, "ccounter");
+ for(i=0;i<pp->clumpsinobj;++i)
+ {
+ tsize=pp->ci[ i * CCOL_NUMCOLS + CCOL_NUM ];
+ clumpsmed[i]=gal_data_alloc(NULL, p->values->type, 1, &tsize, NULL,
+ 0, p->cp.minmapsize, NULL, NULL, NULL);
+ }
+ }
+
+
+ /* Parse each contiguous patch of memory covered by this object. */
+ while( pp->start_end_inc[0] + increment <= pp->start_end_inc[1] )
+ {
+ /* Set the contiguous range to parse. The pixel-to-pixel counting
+ along the fastest dimension will be done over the `O' pointer. */
+ V = pp->st_v + increment;
+ if(p->clumps) C = pp->st_c + increment;
+ OO = ( O = pp->st_o + increment ) + pp->tile->dsize[ndim-1];
+
+ /* Parse the next contiguous region of this tile. */
+ do
+ {
+ /* If this pixel belongs to the requested object, then do the
+ processing. `hasblank' is constant, so when the values doesn't
+ have any blank values, the `isnan' will never be checked. */
+ if( *O==pp->object && !( p->hasblank && isnan(*V) ) )
+ {
+ /* Copy the value for the whole object. */
+ memcpy( gal_data_ptr_increment(objmed->array, counter++,
+ p->values->type), V,
+ gal_type_sizeof(p->values->type) );
+
+ /* We are also on a clump. */
+ if(p->clumps && *C>0)
+ memcpy( gal_data_ptr_increment(clumpsmed[*C-1]->array,
+ ccounter[*C-1]++,
+ p->values->type), V,
+ gal_type_sizeof(p->values->type) );
+ }
+
+ /* Increment the other pointers. */
+ ++V;
+ if(p->clumps) ++C;
+ }
+ while(++O<OO);
+
+ /* Increment to the next contiguous region of this tile. */
+ increment += ( gal_tile_block_increment(p->objects, dsize,
+ num_increment++, NULL) );
+ }
+
+
+ /* Calculate the final medians for objects. */
+ median=gal_data_copy_to_new_type_free(gal_statistics_median(objmed, 1),
+ GAL_TYPE_FLOAT64);
+ pp->oi[OCOL_MEDIAN]=*((double *)(median->array));
+ gal_data_free(objmed);
+ gal_data_free(median);
+
+
+ /* Calculate the median for clumps. */
+ if(p->clumps)
+ {
+ for(i=0;i<pp->clumpsinobj;++i)
+ {
+ ci=&pp->ci[ i * CCOL_NUMCOLS ];
+ median=gal_statistics_median(clumpsmed[i], 1);
+ median=gal_data_copy_to_new_type_free(median, GAL_TYPE_FLOAT64);
+ ci[ CCOL_MEDIAN ] = ( *((double *)(median->array))
+ - (ci[ CCOL_RIV_SUM ]/ci[ CCOL_RIV_NUM ]) );
+ gal_data_free(clumpsmed[i]);
+ gal_data_free(median);
+ }
+ free(clumpsmed);
+ free(ccounter);
+ }
+}
diff --git a/bin/convertt/jpeg.h b/bin/mkcatalog/parse.h
similarity index 67%
rename from bin/convertt/jpeg.h
rename to bin/mkcatalog/parse.h
index 14a9310..5dc0238 100644
--- a/bin/convertt/jpeg.h
+++ b/bin/mkcatalog/parse.h
@@ -1,11 +1,11 @@
/*********************************************************************
-ConvertType - Convert between various types of files.
-ConvertType is part of GNU Astronomy Utilities (Gnuastro) package.
+MakeCatalog - Make a catalog from an input and labeled image.
+MakeCatalog is part of GNU Astronomy Utilities (Gnuastro) package.
Original author:
Mohammad Akhlaghi <mohammad@akhlaghi.org>
Contributing author(s):
-Copyright (C) 2015-2018, Free Software Foundation, Inc.
+Copyright (C) 2018, Free Software Foundation, Inc.
Gnuastro is free software: you can redistribute it and/or modify it
under the terms of the GNU General Public License as published by the
@@ -20,19 +20,19 @@ General Public License for more details.
You should have received a copy of the GNU General Public License
along with Gnuastro. If not, see <http://www.gnu.org/licenses/>.
**********************************************************************/
-#ifndef JPEG_H
-#define JPEG_H
+#ifndef PARSE_H
+#define PARSE_H
-int
-nameisjpeg(char *name);
+void
+parse_initialize(struct mkcatalog_passparams *pp);
-int
-nameisjpegsuffix(char *name);
+void
+parse_objects(struct mkcatalog_passparams *pp);
-size_t
-jpeg_read_to_ll(char *filename, gal_data_t **list, size_t minmapsize);
+void
+parse_clumps(struct mkcatalog_passparams *pp);
void
-jpeg_write(struct converttparams *p);
+parse_median(struct mkcatalog_passparams *pp);
#endif
diff --git a/bin/mkcatalog/ui.c b/bin/mkcatalog/ui.c
index 2543a0e..46b9eeb 100644
--- a/bin/mkcatalog/ui.c
+++ b/bin/mkcatalog/ui.c
@@ -29,8 +29,10 @@ along with Gnuastro. If not, see
<http://www.gnu.org/licenses/>.
#include <string.h>
#include <inttypes.h>
+#include <gnuastro/wcs.h>
#include <gnuastro/fits.h>
#include <gnuastro/blank.h>
+#include <gnuastro/array.h>
#include <gnuastro/threads.h>
#include <gnuastro/dimension.h>
#include <gnuastro/arithmetic.h>
@@ -118,14 +120,15 @@ ui_initialize_options(struct mkcatalogparams *p,
cp->coptions = gal_commonopts_options;
/* Specific to this program. */
+ p->medstd = NAN;
p->sfmagnsigma = NAN;
p->sfmagarea = NAN;
p->upnsigma = NAN;
p->zeropoint = NAN;
- p->threshold = NAN;
p->upsigmaclip[0] = NAN;
p->upsigmaclip[1] = NAN;
-
+ p->checkupperlimit[0] = GAL_BLANK_INT32;
+ p->checkupperlimit[1] = GAL_BLANK_INT32;
/* Modify common options. */
for(i=0; !gal_options_is_last(&cp->coptions[i]); ++i)
@@ -137,6 +140,9 @@ ui_initialize_options(struct mkcatalogparams *p,
case GAL_OPTIONS_KEY_TYPE:
case GAL_OPTIONS_KEY_SEARCHIN:
case GAL_OPTIONS_KEY_IGNORECASE:
+ case GAL_OPTIONS_KEY_WORKOVERCH:
+ case GAL_OPTIONS_KEY_INTERPNUMNGB:
+ case GAL_OPTIONS_KEY_INTERPONLYBLANK:
cp->coptions[i].flags=OPTION_HIDDEN;
cp->coptions[i].mandatory=GAL_OPTIONS_NOT_MANDATORY;
break;
@@ -145,15 +151,6 @@ ui_initialize_options(struct mkcatalogparams *p,
cp->coptions[i].mandatory=GAL_OPTIONS_MANDATORY;
break;
}
-
- /* Select by group. */
- switch(cp->coptions[i].group)
- {
- case GAL_OPTIONS_GROUP_TESSELLATION:
- cp->coptions[i].doc=NULL; /* Necessary to remove title. */
- cp->coptions[i].flags=OPTION_HIDDEN;
- break;
- }
}
}
@@ -188,10 +185,10 @@ parse_opt(int key, char *arg, struct argp_state *state)
/* Read the non-option tokens (arguments): */
case ARGP_KEY_ARG:
- if(p->inputname)
+ if(p->objectsfile)
argp_error(state, "only one argument (input file) should be given");
else
- p->inputname=arg;
+ p->objectsfile=arg;
break;
@@ -252,6 +249,91 @@ ui_column_codes_ll(struct argp_option *option, char *arg,
+/* Prepare the upper-limit distribution parameters. */
+void *
+ui_check_upperlimit(struct argp_option *option, char *arg,
+ char *filename, size_t lineno, void *params)
+{
+ size_t i;
+ char *str;
+ double *d;
+ gal_data_t *raw;
+ struct mkcatalogparams *p=(struct mkcatalogparams *)params;
+
+ /* Write. */
+ if(lineno==-1)
+ {
+ if(p->checkupperlimit[1]==GAL_BLANK_INT32)
+ {
+ if( asprintf(&str, "%d", p->checkupperlimit[0])<0 )
+ error(EXIT_FAILURE, 0, "%s: asprintf allocation", __func__);
+ }
+ else
+ if( asprintf(&str, "%d,%d", p->checkupperlimit[0],
+ p->checkupperlimit[1])<0 )
+ error(EXIT_FAILURE, 0, "%s: asprintf allocation", __func__);
+ return str;
+ }
+
+ /* Read */
+ else
+ {
+ /* If the option is already set, just return. */
+ if(option->set) return NULL;
+
+ /* Read the list of numbers as an array. */
+ raw=gal_options_parse_list_of_numbers(arg, filename, lineno);
+
+ /* Make sure there is at most only two numbers given. */
+ if(raw->size>2)
+ error_at_line(EXIT_FAILURE, 0, filename, lineno, "`%s' (value to "
+ "`--%s') contains %zu numbers, but only one or two "
+ "are acceptable.\n\n"
+ "With this option MakeCatalog will write all the "
+ "positions and values of the random distribution for "
+ "one particular labeled region into a table. The "
+ "given value(s) is(are) the label identifier.\n\n"
+ "With one value the distribution for an object will "
+ "be printed: the givne number will be interpretted as "
+ "the requested object's label. With two values, the "
+ "distribution for a specific clump will be written. "
+ "The first will be interpretted as the clump's host "
+ "object label and the second as the clump's label "
+ "within the object", arg, option->name, raw->size);
+
+ /* Make sure the given values are integers and that they are larger
+ than zero. */
+ d=raw->array;
+ for(i=0;i<raw->size;++i)
+ {
+ if( ceil(d[i]) != d[i])
+ error_at_line(EXIT_FAILURE, 0, filename, lineno, "%g (value "
+ "number %zu given to `--%s') is not an "
+ "integer. The value(s) to this option are "
+ "object/clump labels/identifiers, so they must be "
+ "integers", d[i], i+1, option->name);
+ if( d[i]<=0 )
+ error_at_line(EXIT_FAILURE, 0, filename, lineno, "%g (value "
+ "number %zu given to `--%s') is not positive. "
+ "The value(s) to this option are object/clump "
+ "labels/identifiers, so they must be positive "
+ "integers", d[i], i+1, option->name);
+ }
+
+ /* Write the values in. */
+ p->checkupperlimit[0] = d[0];
+ p->checkupperlimit[1] = raw->size==2 ? d[1] : GAL_BLANK_INT32;
+
+ /* For no un-used variable warning. This function doesn't need the
+ pointer.*/
+ return NULL;
+ }
+}
+
+
+
+
+
@@ -271,12 +353,58 @@ ui_column_codes_ll(struct argp_option *option, char *arg,
/*************** Sanity Check *******************/
/**************************************************************/
/* Read and check ONLY the options. When arguments are involved, do the
- check in `ui_check_options_and_arguments'.
+ check in `ui_check_options_and_arguments'. */
static void
ui_read_check_only_options(struct mkcatalogparams *p)
{
+ float tmp;
+ size_t one=1;
+ char *tailptr;
+
+ /* If an upper-limit check table is requested with a specific clump, but
+ no clump catalog has been requested, then abort and inform the
+ user. */
+ if( p->checkupperlimit[1]!=GAL_BLANK_INT32 && p->clumpscat==0 )
+ error(EXIT_FAILURE, 0, "no clumps catalog is requested, hence "
+ "`--checkupperlimit' is only available for objects (one value "
+ "must be given to it).\n\n"
+ "To ask for a clumps catalog, please append `--clumpscat' to the "
+ "command calling MakeCatalog.\n\n"
+ "If you want the upperlimit check table for an object, only give "
+ "one value (the object's label) to `--checkupperlimit'.");
+
+ /* See if `--skyin' is a filename or a value. When the string is ONLY a
+ number (and nothing else), `tailptr' will point to the end of the
+ string (`\0'). */
+ if(p->skyfile)
+ {
+ tmp=strtod(p->skyfile, &tailptr);
+ if(*tailptr=='\0')
+ {
+ /* Allocate the data structure. */
+ p->sky=gal_data_alloc(NULL, GAL_TYPE_FLOAT32, 1, &one, NULL, 0, -1,
+ NULL, NULL, NULL);
+
+ /* Write the value inside it. */
+ *((float *)(p->sky->array))=tmp;
+ }
+ }
+
+ /* Similar to the case for Sky above. */
+ if(p->stdfile)
+ {
+ tmp=strtod(p->stdfile, &tailptr);
+ if(*tailptr=='\0')
+ {
+ /* Allocate the data structure. */
+ p->std=gal_data_alloc(NULL, GAL_TYPE_FLOAT32, 1, &one, NULL, 0, -1,
+ NULL, NULL, NULL);
+
+ /* Write the value inside it. */
+ *((float *)(p->std->array))=tmp;
+ }
+ }
}
-*/
@@ -284,11 +412,11 @@ ui_read_check_only_options(struct mkcatalogparams *p)
static void
ui_check_options_and_arguments(struct mkcatalogparams *p)
{
- /* Make sure an input file name was given and if it was a FITS file, that
- a HDU is also given. */
- if(p->inputname)
+ /* Make sure the main input file name (for the object labels) was given
+ and if it was a FITS file, that a HDU is also given. */
+ if(p->objectsfile)
{
- if( gal_fits_name_is_fits(p->inputname) && p->cp.hdu==NULL )
+ if( gal_fits_name_is_fits(p->objectsfile) && p->cp.hdu==NULL )
error(EXIT_FAILURE, 0, "no HDU specified. When the input is a FITS "
"file, a HDU must also be specified, you can use the `--hdu' "
"(`-h') option and give it the HDU number (starting from "
@@ -321,6 +449,48 @@ ui_check_options_and_arguments(struct mkcatalogparams *p)
/**************************************************************/
/*************** Preparations *******************/
/**************************************************************/
+/* If the user hasn't explicitly specified a filename for input,
+ MakeCatalog will use other given file names. */
+static void
+ui_set_filenames(struct mkcatalogparams *p)
+{
+ p->usedclumpsfile = p->clumpsfile ? p->clumpsfile : p->objectsfile;
+
+ p->usedvaluesfile = p->valuesfile ? p->valuesfile : p->objectsfile;
+
+ p->usedskyfile = ( p->skyfile
+ ? p->skyfile
+ : ( p->valuesfile ? p->valuesfile : p->objectsfile ) );
+
+ p->usedstdfile = ( p->stdfile
+ ? p->stdfile
+ : ( p->valuesfile ? p->valuesfile : p->objectsfile ) );
+}
+
+
+
+
+
+/* The clumps and objects images must be integer type, so we'll use this
+ function to avoid having to write the error message two times. */
+static void
+ui_check_type_int(char *filename, char *hdu, uint8_t type)
+{
+ if( type==GAL_TYPE_FLOAT32 || type==GAL_TYPE_FLOAT64 )
+ error(EXIT_FAILURE, 0, "%s (hdu: %s): type %s not acceptable as "
+ "labels input. labeled images must have an integer datatype.\n\n"
+ "If you are sure the extension contains only integer values but "
+ "it is just stored in a floating point container, you can "
+ "put it in an integer container with Gnuastro's Arithmetic "
+ "program using this command:\n\n"
+ " $ astarithmetic %s int32 -h%s", filename, hdu,
+ gal_type_name(type, 1), filename, hdu);
+}
+
+
+
+
+
/* If a WCS structure is present, then read its basic information to use in
the table meta-data. */
static void
@@ -329,24 +499,28 @@ ui_wcs_info(struct mkcatalogparams *p)
char *c;
size_t i;
+ /* Read the WCS meta-data. */
+ p->objects->wcs=gal_wcs_read(p->objectsfile, p->cp.hdu, 0, 0,
+ &p->objects->nwcs);
+
/* Read the basic WCS information. */
- if(p->input->wcs)
+ if(p->objects->wcs)
{
/* Allocate space for the array of strings. */
errno=0;
- p->ctype=malloc(p->input->ndim * sizeof *p->ctype);
+ p->ctype=malloc(p->objects->ndim * sizeof *p->ctype);
if(p->ctype==NULL)
error(EXIT_FAILURE, 0, "%s: %zu bytes for `p->ctype'", __func__,
- p->input->ndim * sizeof *p->ctype);
+ p->objects->ndim * sizeof *p->ctype);
/* Fill in the values. */
- for(i=0;i<p->input->ndim;++i)
+ for(i=0;i<p->objects->ndim;++i)
{
/* CTYPE might contain `-' characters, we just want the first
non-dash characters. The loop will either stop either at the end
or where there is a dash. So we can just replace it with an
end-of-string character. */
- gal_checkset_allocate_copy(p->input->wcs->ctype[i], &p->ctype[i]);
+ gal_checkset_allocate_copy(p->objects->wcs->ctype[i], &p->ctype[i]);
c=p->ctype[i]; while(*c!='\0' && *c!='-') ++c;
*c='\0';
}
@@ -357,354 +531,569 @@ ui_wcs_info(struct mkcatalogparams *p)
+/* The only mandatory input is the objects image, so first read that and
+ make sure its type is correct. */
static void
-ui_preparations_read_inputs(struct mkcatalogparams *p)
+ui_read_labels(struct mkcatalogparams *p)
{
- size_t one=1;
- int readclumps=0;
- char *namestypes, **strarr=NULL;
- gal_data_t *zero, *key=gal_data_array_calloc(1);
- char *skyfile=p->skyfile ? p->skyfile : p->inputname;
- char *stdfile=p->stdfile ? p->stdfile : p->inputname;
- char *clumpsfile=p->clumpsfile ? p->clumpsfile : p->inputname;
- char *objectsfile=p->objectsfile ? p->objectsfile : p->inputname;
+ char *msg;
+ gal_data_t *tmp, *keys=gal_data_array_calloc(2);
+ /* Read it into memory. */
+ p->objects = gal_array_read_one_ch(p->objectsfile, p->cp.hdu,
+ p->cp.minmapsize);
- /* Read the input image. */
- p->input=gal_fits_img_read_to_type(p->inputname, p->cp.hdu,
- GAL_TYPE_FLOAT32, p->cp.minmapsize, 0,0);
+ /* Make sure it has an integer type. */
+ ui_check_type_int(p->objectsfile, p->cp.hdu, p->objects->type);
- /* Read basic WCS information for final table meta-data. */
- ui_wcs_info(p);
+
+ /* Convert it to `int32' type (if it already isn't). */
+ p->objects=gal_data_copy_to_new_type_free(p->objects, GAL_TYPE_INT32);
/* Currently MakeCatalog is only implemented for 2D images. */
- if(p->input->ndim!=2 && p->input->ndim!=3)
+ if(p->objects->ndim!=2 && p->objects->ndim!=3)
error(EXIT_FAILURE, 0, "%s (hdu %s) has %zu dimensions, MakeCatalog "
- "currently only supports 2D inputs", p->inputname, p->cp.hdu,
- p->input->ndim);
-
-
- /* See if the input has blank pixels and set the flags appropriately. */
- p->hasblank = gal_blank_present(p->input, 1);
-
-
- /* Read the object label image and check its size. */
- p->objects = gal_fits_img_read(objectsfile, p->objectshdu,
- p->cp.minmapsize, 0, 0);
- if( gal_data_dsize_is_different(p->input, p->objects) )
- error(EXIT_FAILURE, 0, "`%s' (hdu: %s) and `%s' (hdu: %s) have a"
- "different dimension/size", objectsfile, p->objectshdu,
- p->inputname, p->cp.hdu);
-
-
- /* Read the Sky image and check its size. */
- p->sky=gal_fits_img_read_to_type(skyfile, p->skyhdu, GAL_TYPE_FLOAT32,
- p->cp.minmapsize, 0, 0);
- if( gal_data_dsize_is_different(p->input, p->sky) )
- error(EXIT_FAILURE, 0, "`%s' (hdu: %s) and `%s' (hdu: %s) have a"
- "different dimension/size", skyfile, p->skyhdu, p->inputname,
- p->cp.hdu);
-
+ "currently only supports 2D or 3D datasets", p->objectsfile,
+ p->cp.hdu, p->objects->ndim);
+
+ /* See if the total number of objects is given in the header keywords. */
+ keys[0].name="NUMLABS";
+ keys[0].type=GAL_TYPE_SIZE_T;
+ keys[0].array=&p->numobjects;
+ gal_fits_key_read(p->objectsfile, p->cp.hdu, keys, 0, 0);
+ if(keys[0].status) /* status!=0: the key couldn't be read by CFITSIO. */
+ {
+ tmp=gal_statistics_maximum(p->objects);
+ p->numobjects=*((int32_t *)(tmp->array)); /*numobjects is in int32_t.*/
+ gal_data_free(tmp);
+ }
- /* Read the Sky standard deviation image and check its size. */
- p->std=gal_fits_img_read_to_type(stdfile, p->stdhdu, GAL_TYPE_FLOAT32,
- p->cp.minmapsize, 0, 0);
- if( gal_data_dsize_is_different(p->input, p->std) )
- error(EXIT_FAILURE, 0, "`%s' (hdu: %s) and `%s' (hdu: %s) have a"
- "different dimension/size", stdfile, p->stdhdu, p->inputname,
- p->cp.hdu);
+ /* If there were no objects in the input, then inform the user with an
+ error (it is pointless to build a catalog). */
+ if(p->numobjects==0)
+ error(EXIT_FAILURE, 0, "no object labels (non-zero pixels) in "
+ "%s (hdu %s). To make a catalog, labeled regions must be defined",
+ p->objectsfile, p->cp.hdu);
- /* If an upper-limit mask image was given, read it. */
- if(p->upmaskfile)
- {
- /* Read the mask image. */
- p->upmask = gal_fits_img_read(p->upmaskfile, p->upmaskhdu,
- p->cp.minmapsize, 0, 0);
- if( gal_data_dsize_is_different(p->input, p->upmask) )
- error(EXIT_FAILURE, 0, "`%s' (hdu: %s) and `%s' (hdu: %s) have a"
- "different dimension/size", p->upmaskfile, p->upmaskhdu,
- p->inputname, p->cp.hdu);
-
- /* If it isn't an integer type, report an error, otherwise, convert
- it to a uint8_t: with a 1 for all non-zero pixels and 0 for zero
- pixels. */
- zero=gal_data_alloc(NULL, GAL_TYPE_UINT8, 1, &one, NULL, 1, -1,
- NULL, NULL, NULL);
- p->upmask=gal_arithmetic(GAL_ARITHMETIC_OP_NE,
- ( GAL_ARITHMETIC_INPLACE | GAL_ARITHMETIC_FREE
- | GAL_ARITHMETIC_NUMOK ), p->upmask, zero);
- }
+ /* See if the labels image has blank pixels and set the flags
+ appropriately. */
+ p->hasblank = gal_blank_present(p->objects, 1);
- /* Check to see if the objects extension has a `WCLUMPS' keyword and that
- its value is 'yes', `y', or `1'. */
- key->name="WCLUMPS";
- key->type=GAL_TYPE_STRING;
- gal_fits_key_read(objectsfile, p->objectshdu, key, 0, 0);
- if(key->status==KEY_NO_EXIST) readclumps=0;
- else
- {
- if(key->status)
- gal_fits_io_error(key->status, "CFITSIO error while reading "
- "WCLUMPS keyword");
- else
- {
- strarr=key->array;
- if( !strcasecmp(strarr[0], "yes") || !strcasecmp(strarr[0], "y")
- || !strcmp(strarr[0], "1") ) readclumps=1;
- }
- }
+ /* Prepare WCS information for final table meta-data. */
+ ui_wcs_info(p);
/* Read the clumps array if necessary. */
- if(readclumps)
+ if(p->clumpscat)
{
- /* Make sure the user did indeed give the clumps HDU. */
+ /* Make sure the HDU is also given. */
if(p->clumpshdu==NULL)
- error(EXIT_FAILURE, 0, "no `--clumpshdu' given! The WCLUMPS keyword "
- "in %s (hdu: %s) has a value of `%s', so MakeCatalog expects "
- "a clump image.\n\nYou can use the optional `--clumpsfile' "
- "option to give the filename and the mandatory `--clumpshdu' "
- "to specify the extension. If `--clumpsfile' is not given, "
- "MakeCatalog will look into the input file for the given "
- "extension. Alternatively, you can modify/remove this "
- "keyword using Gnuastro's Fits program, please run `$ info "
- "astfits' for more information (press `SPACE' to go down and "
- "`q' to return to the command-line).", objectsfile,
- p->objectshdu, strarr[0]);
-
- /* Read the clumps image and check its size. */
- p->clumps = gal_fits_img_read(clumpsfile, p->clumpshdu,
- p->cp.minmapsize, 0, 0);
- if( gal_data_dsize_is_different(p->input, p->std) )
+ error(EXIT_FAILURE, 0, "%s: no HDU/extension provided for the "
+ "CLUMPS dataset. Please use the `--clumpshdu' option to "
+ "give a specific HDU using its number (counting from zero) "
+ "or name. If the dataset is in another file, please use "
+ "`--clumpsfile' to give the filename. If you don't want any "
+ "clumps catalog output, remove the `--clumpscat' option from "
+ "the command-line or give it a value of zero in a "
+ "configuration file", p->usedclumpsfile);
+
+ /* Read the clumps image. */
+ p->clumps = gal_array_read_one_ch(p->usedclumpsfile, p->clumpshdu,
+ p->cp.minmapsize);
+
+ /* Check its size. */
+ if( gal_data_dsize_is_different(p->objects, p->clumps) )
error(EXIT_FAILURE, 0, "`%s' (hdu: %s) and `%s' (hdu: %s) have a"
- "different dimension/size", clumpsfile, p->clumpshdu,
- p->inputname, p->cp.hdu);
- }
+ "different dimension/size", p->usedclumpsfile, p->clumpshdu,
+ p->objectsfile, p->cp.hdu);
+ /* Check its type. */
+ ui_check_type_int(p->usedclumpsfile, p->clumpshdu, p->clumps->type);
+ p->clumps=gal_data_copy_to_new_type_free(p->clumps, GAL_TYPE_INT32);
- /* Make sure the object and clump label images have an integer type, then
- to be safe that they have the correct integer type, run
- `gal_data_copy_to_new_type' on them. */
- if( p->objects->type==GAL_TYPE_FLOAT32
- || p->objects->type==GAL_TYPE_FLOAT64
- || (p->clumps && ( p->clumps->type==GAL_TYPE_FLOAT32
- || p->clumps->type==GAL_TYPE_FLOAT64 ) ) )
- {
- if(p->clumps)
+ /* See if there are keywords to help in finding the number. */
+ keys[0].next=&keys[1];
+ keys[0].status=keys[1].status=0;
+ keys[0].name="CLUMPSN"; keys[1].name="NUMLABS";
+ keys[0].type=GAL_TYPE_FLOAT32; keys[1].type=GAL_TYPE_SIZE_T;
+ keys[0].array=&p->clumpsn; keys[1].array=&p->numclumps;
+ gal_fits_key_read(p->usedclumpsfile, p->clumpshdu, keys, 0, 0);
+ if(keys[0].status) p->clumpsn=NAN;
+ if(keys[1].status)
{
- if( asprintf(&namestypes, "However, `%s' (hdu: %s) and `%s' "
- "(hdu: %s) have types of `%s' and `%s' respectively",
- objectsfile, p->objectshdu, clumpsfile, p->clumpshdu,
- gal_type_name(p->objects->type, 1),
- gal_type_name(p->clumps->type, 1) )<0 )
+ if( asprintf(&msg, "%s (hdu: %s): couldn't find/read `NUMLABS' in "
+ "the header keywords, see CFITSIO error above. The "
+ "clumps image must have the total number of clumps "
+ "(irrespective of how many objects there are in the "
+ "image) in this header keyword", p->usedclumpsfile,
+ p->clumpshdu)<0 )
error(EXIT_FAILURE, 0, "%s: asprintf allocation", __func__);
+ gal_fits_io_error(keys[1].status, msg);
}
- else
+
+ /* If there were no clumps, then free the clumps array and set it to
+ NULL, so for the rest of the processing, MakeCatalog things that
+ no clumps image was given. */
+ if(p->numclumps==0)
{
- if( asprintf(&namestypes, "However, %s (hdu: %s) has a type of %s",
- objectsfile, p->objectshdu,
- gal_type_name(p->objects->type, 1))<0 )
- error(EXIT_FAILURE, 0, "%s: asprintf allocation", __func__);
+ /* Just as a sanity check, see if there are any clumps (positive
+ valued pixels) in the array. If there are, then `NUMCLUMPS'
+ wasn't set properly and we should abort with an error. */
+ tmp=gal_statistics_maximum(p->clumps);
+ if( *((int32_t *)(p->clumps->array))>0 )
+ error(EXIT_FAILURE, 0, "%s (hdu: %s): the `NUMCLUMPS' header "
+ "keyword has a value of zero, but there are positive "
+ "pixels in the array, showing that there are clumps in "
+ "image. This is a wrong usage of the `NUMCLUMPS' keyword."
+ "It must contain the total number of clumps (irrespective "
+ "of how many objects there are). Please correct this issue "
+ "and run MakeCatalog again", p->usedclumpsfile,
+ p->clumpshdu);
+
+ /* Since there are no clumps, we won't bother creating a clumps
+ catalog and from this step onward, we'll act as if no clumps
+ catalog was requested. In order to not confuse the user in the
+ end, we'll print a warning first. */
+ fprintf(stderr, "WARNING: %s (hdu %s): there are no clumps "
+ "in the image, therefore no clumps catalog will be "
+ "created.\n", p->usedclumpsfile, p->clumpshdu);
+ gal_data_free(p->clumps);
+ p->clumps=NULL;
}
- error(EXIT_FAILURE, 0, "labeled images (for objects or clumps) must "
- "have an integer datatype. %s.\n\n"
- "If you are sure the images contain only integer values but "
- "are just stored in a floating point container, you can "
- "put them in an integer container with Gnuastro's Arithmetic "
- "program using a command like below:\n\n"
- " $ astarithmetic img.fits int32", namestypes);
}
- p->objects=gal_data_copy_to_new_type_free(p->objects, GAL_TYPE_INT32);
- if(p->clumps)
- p->clumps=gal_data_copy_to_new_type_free(p->clumps, GAL_TYPE_INT32);
/* Clean up. */
- key->name=NULL;
- gal_data_array_free(key, 1, 1);
+ keys[0].name=keys[1].name=NULL;
+ keys[0].array=keys[1].array=NULL;
+ gal_data_array_free(keys, 2, 1);
}
-/* The input images can have extensions to speed up the processing. */
+/* See which inputs are necessary. Ultimate, there are only three extra
+ inputs: a values image, a sky image and a sky standard deviation
+ image. However, there are many raw column measurements. So to keep
+ things clean, we'll just put a value of `1' in the three `values', `sky'
+ and `std' pointers everytime a necessary input is found. */
static void
-ui_preparations_read_keywords(struct mkcatalogparams *p)
+ui_necessary_inputs(struct mkcatalogparams *p, int *values, int *sky,
+ int *std)
{
- char *msg;
- gal_data_t *tmp;
- gal_data_t *keys=gal_data_array_calloc(2);
- char *stdfile=p->stdfile ? p->stdfile : p->inputname;
- char *clumpsfile=p->clumpsfile ? p->clumpsfile : p->inputname;
- char *objectsfile=p->objectsfile ? p->objectsfile : p->inputname;
+ size_t i;
+
+ if(p->upperlimit) *values=1;
+
+ /* Go over all the object columns. Note that the objects and clumps (if
+ the `--clumpcat' option is given) inputs are mandatory and it is not
+ necessary to specify it here. */
+ for(i=0; i<OCOL_NUMCOLS; ++i)
+ if(p->oiflag[i])
+ switch(i)
+ {
+ case OCOL_NUMALL: /* Only object labels. */ break;
+ case OCOL_NUM: *values = 1; break;
+ case OCOL_SUM: *values = 1; break;
+ case OCOL_SUM_VAR: *values = *std = 1; break;
+ case OCOL_MEDIAN: *values = 1; break;
+ case OCOL_VX: *values = 1; break;
+ case OCOL_VY: *values = 1; break;
+ case OCOL_VZ: *values = 1; break;
+ case OCOL_VXX: *values = 1; break;
+ case OCOL_VYY: *values = 1; break;
+ case OCOL_VXY: *values = 1; break;
+ case OCOL_SUMSKY: *sky = 1; break;
+ case OCOL_SUMSTD: *std = 1; break;
+ case OCOL_SUMWHT: *values = 1; break;
+ case OCOL_NUMWHT: *values = 1; break;
+ case OCOL_GX: /* Only object labels. */ break;
+ case OCOL_GY: /* Only object labels. */ break;
+ case OCOL_GZ: /* Only object labels. */ break;
+ case OCOL_GXX: /* Only object labels. */ break;
+ case OCOL_GYY: /* Only object labels. */ break;
+ case OCOL_GXY: /* Only object labels. */ break;
+ case OCOL_UPPERLIMIT_B: *values = 1; break;
+ case OCOL_UPPERLIMIT_S: *values = 1; break;
+ case OCOL_UPPERLIMIT_Q: *values = 1; break;
+ case OCOL_UPPERLIMIT_SKEW: *values = 1; break;
+ case OCOL_C_NUMALL: /* Only clump labels. */ break;
+ case OCOL_C_NUM: *values = 1; break;
+ case OCOL_C_SUM: *values = 1; break;
+ case OCOL_C_VX: *values = 1; break;
+ case OCOL_C_VY: *values = 1; break;
+ case OCOL_C_VZ: *values = 1; break;
+ case OCOL_C_GX: /* Only clump labels. */ break;
+ case OCOL_C_GY: /* Only clump labels. */ break;
+ case OCOL_C_GZ: /* Only clump labels. */ break;
+ case OCOL_C_SUMWHT: *values = 1; break;
+ case OCOL_C_NUMWHT: *values = 1; break;
+ default:
+ error(EXIT_FAILURE, 0, "%s: a bug! Please contact us at %s to "
+ "fix the problem. The code %zu is not a recognized "
+ "intermediate OBJECT columns", __func__, PACKAGE_BUGREPORT,
+ i);
+ }
+
+ /* Check the clump elements also. */
+ if(p->clumps)
+ for(i=0; i<CCOL_NUMCOLS; ++i)
+ if(p->ciflag[i])
+ switch(i)
+ {
+ case CCOL_NUMALL: /* Only clump labels. */ break;
+ case CCOL_NUM: *values = 1; break;
+ case CCOL_SUM: *values = 1; break;
+ case CCOL_SUM_VAR: *values = *std = 1; break;
+ case CCOL_MEDIAN: *values = 1; break;
+ case CCOL_RIV_NUM: /* Only clump labels. */ break;
+ case CCOL_RIV_SUM: *values = 1; break;
+ case CCOL_RIV_SUM_VAR: *values = *std = 1; break;
+ case CCOL_VX: *values = 1; break;
+ case CCOL_VY: *values = 1; break;
+ case CCOL_VZ: *values = 1; break;
+ case CCOL_VXX: *values = 1; break;
+ case CCOL_VYY: *values = 1; break;
+ case CCOL_VXY: *values = 1; break;
+ case CCOL_SUMSKY: *sky = 1; break;
+ case CCOL_SUMSTD: *std = 1; break;
+ case CCOL_SUMWHT: *values = 1; break;
+ case CCOL_NUMWHT: *values = 1; break;
+ case CCOL_GX: /* Only clump labels. */ break;
+ case CCOL_GY: /* Only clump labels. */ break;
+ case CCOL_GZ: /* Only clump labels. */ break;
+ case CCOL_GXX: /* Only clump labels. */ break;
+ case CCOL_GYY: /* Only clump labels. */ break;
+ case CCOL_GXY: /* Only clump labels. */ break;
+ case CCOL_UPPERLIMIT_B: *values = 1; break;
+ case CCOL_UPPERLIMIT_S: *values = 1; break;
+ case CCOL_UPPERLIMIT_Q: *values = 1; break;
+ case CCOL_UPPERLIMIT_SKEW: *values = 1; break;
+ default:
+ error(EXIT_FAILURE, 0, "%s: a bug! Please contact us at %s to "
+ "fix the problem. The code %zu is not a recognized "
+ "intermediate CLUMP column", __func__, PACKAGE_BUGREPORT,
+ i);
+ }
+}
- /* Read the keywords from the standard deviation image. */
- keys[0].next=&keys[1];
- keys[0].name="MINSTD"; keys[1].name="MEDSTD";
- keys[0].type=GAL_TYPE_FLOAT32; keys[1].type=GAL_TYPE_FLOAT32;
- keys[0].array=&p->minstd; keys[1].array=&p->medstd;
- gal_fits_key_read(stdfile, p->stdhdu, keys, 0, 0);
- /* If the two keywords couldn't be read, calculate them. */
- if(keys[0].status)
- {
- /* Calculate the minimum STD. */
- tmp=gal_statistics_minimum(p->std);
- p->minstd=*((float *)(tmp->array));
- gal_data_free(tmp);
- }
- if(keys[1].status)
- {
- /* Calculate the median STD. */
- tmp=gal_statistics_median(p->std, 0);
- p->medstd=*((float *)(tmp->array));
- gal_data_free(tmp);
- /* Alert the user if it wasn't calculated from a header keyword. */
- fprintf(stderr, "---------------\n"
- "Warning: Could not find the `MEDSTD' keyword in `%s' "
- "(hdu: %s). The median standard deviation is thus found on "
- "the (interpolated) standard deviation image. NoiseChisel "
- "finds the median before interpolation which is more "
- "accurate. Ho the reported value in the final catalog will "
- "not be accurate: it will depend on how many tiles were "
- "blank and their spatial position relative to the non-blank "
- "ones.\n"
- "---------------\n", stdfile, p->stdhdu);
- }
- p->cpscorr = p->minstd>1 ? 1.0f : p->minstd;
+/* When the Sky and its standard deviation are given as tiles, we need to
+ define a tile structure. */
+static void
+ui_preparation_check_size_read_tiles(struct mkcatalogparams *p,
+ gal_data_t *in, char *filename,
+ char *hdu)
+{
+ struct gal_tile_two_layer_params *tl=&p->cp.tl;
- /* Read the keywords from the objects image. */
- keys[0].name="DETSN"; keys[1].name="NUMLABS";
- keys[0].type=GAL_TYPE_FLOAT32; keys[1].type=GAL_TYPE_SIZE_T;
- keys[0].array=&p->detsn; keys[1].array=&p->numobjects;
- gal_fits_key_read(objectsfile, p->objectshdu, keys, 0, 0);
- if(keys[0].status) p->detsn=NAN; /* When `status!=0', then */
- if(keys[1].status) /* the key couldn't be read. */
+ /* See if we should treat this dataset as tile values or not. */
+ if( gal_data_dsize_is_different(p->objects, in) )
{
- tmp=gal_statistics_maximum(p->objects);
- p->numobjects=*((int32_t *)(tmp->array)); /*numobjects is in int32_t.*/
- gal_data_free(tmp);
+ /* The `tl' structure is initialized here. But this function may be
+ called multiple times. So, first check if the `tl' structure has
+ already been initialized and if so, don't repeat it. */
+ if(tl->ndim==0)
+ {
+ gal_tile_full_sanity_check(p->objectsfile, p->cp.hdu, p->objects,
+ tl);
+ gal_tile_full_two_layers(p->objects, tl);
+ gal_tile_full_permutation(tl);
+ }
+
+ /* See if the size of the `in' dataset corresponds to the
+ tessellation. */
+ if(in->size!=tl->tottiles)
+ error(EXIT_FAILURE, 0, "%s (hdu: %s): doesn't have the right "
+ "size (%zu elements or pixels).\n\n"
+ "It must either be the same size as `%s' (hdu: `%s'), or "
+ "it must have the same number of elements as the total "
+ "number of tiles in the tessellation (%zu). In the latter "
+ "case, each pixel is assumed to be a fixed value for a "
+ "complete tile.\n\n"
+ "Run with `-P' to see the (tessellation) options/settings "
+ "and their values). For more information on tessellation in "
+ "Gnuastro, please run the following command (use the arrow "
+ "keys for up and down and press `q' to return to the "
+ "command-line):\n\n"
+ " $ info gnuastro tessellation",
+ filename, hdu, in->size, p->objectsfile, p->cp.hdu,
+ tl->tottiles);
}
+}
- /* If there were no objects in the input, then inform the user with an
- error (no catalog was built). */
- if(p->numobjects==0)
- error(EXIT_FAILURE, 0, "no object labels (non-zero pixels) in "
- "%s (hdu %s). To make a catalog, labeled regions must be defined",
- objectsfile, p->objectshdu);
- /* Read the keywords from the clumps image if necessary. */
- if(p->clumps)
+
+/* Subtract `sky' from the input dataset depending on its size (it may be
+ the whole array or a tile-values array).. */
+static void
+ui_subtract_sky(struct mkcatalogparams *p)
+{
+ size_t tid;
+ gal_data_t *tile;
+ float *s, *f, *ff, *skyarr=p->sky->array;
+ struct gal_tile_two_layer_params *tl=&p->cp.tl;
+
+ /* It is the same size as the input or a single value. */
+ if( gal_data_dsize_is_different(p->values, p->sky)==0 || p->sky->size==1)
{
- keys[0].name="CLUMPSN"; keys[1].name="NUMLABS";
- keys[0].type=GAL_TYPE_FLOAT32; keys[1].type=GAL_TYPE_SIZE_T;
- keys[0].array=&p->clumpsn; keys[1].array=&p->numclumps;
- gal_fits_key_read(clumpsfile, p->clumpshdu, keys, 0, 0);
- if(keys[0].status) p->clumpsn=NAN;
- if(keys[1].status)
+ s=p->sky->array;
+ ff = (f=p->values->array) + p->values->size;
+ if(p->sky->size==1) { if(*s!=0.0) do *f-=*s; while(++f<ff); }
+ else do *f-=*s++; while(++f<ff);
+ }
+
+ /* It is the same size as the number of tiles. */
+ else if( tl->tottiles==p->sky->size )
+ {
+ /* Go over all the tiles. */
+ for(tid=0; tid<tl->tottiles; ++tid)
{
- if( asprintf(&msg, "couldn't find/read NUMLABS in the header of "
- "%s (hdu: %s), see error above", clumpsfile,
- p->clumpshdu)<0 )
- error(EXIT_FAILURE, 0, "%s: asprintf allocation", __func__);
- gal_fits_io_error(keys[1].status, msg);
+ /* For easy reading. */
+ tile=&tl->tiles[tid];
+
+ /* Subtract the Sky value from the input image. */
+ GAL_TILE_PARSE_OPERATE(tile, NULL, 0, 0, {*i-=skyarr[tid];});
}
}
+ /* The size must have been checked before, so if control reaches here, we
+ have a bug! */
+ else
+ error(EXIT_FAILURE, 0, "%s: a bug! Please contact us at %s to fix "
+ "the problem. For some reason, the size doesn't match", __func__,
+ PACKAGE_BUGREPORT);
+}
+
- /* If there were no clumps, then free the clumps array and set it to
- NULL, so for the rest of the processing, MakeCatalog things that no
- clumps image was given. */
- if(p->numclumps==0)
+
+
+
+static void
+ui_preparations_read_inputs(struct mkcatalogparams *p)
+{
+ size_t one=1;
+ gal_data_t *zero;
+ gal_data_t *column;
+ int need_values=0, need_sky=0, need_std=0;
+
+ /* See which inputs are necessary. */
+ ui_necessary_inputs(p, &need_values, &need_sky, &need_std);
+
+
+ /* If the values dataset is necessary, read it in and set the units of
+ the columns from it (if it has any). */
+ if(need_values)
{
- gal_data_free(p->clumps);
- p->clumps=NULL;
+ /* Make sure the HDU is also given. */
+ if(p->valueshdu==NULL)
+ error(EXIT_FAILURE, 0, "%s: no HDU/extension provided for the "
+ "VALUES dataset. Atleast one column needs this dataset. "
+ "Please use the `--valueshdu' option to give a specific HDU "
+ "using its number (counting from zero) or name. If the "
+ "dataset is in another file, please use `--valuesfile' to "
+ "give the filename", p->usedvaluesfile);
+
+ /* Read the values dataset. */
+ p->values=gal_array_read_one_ch_to_type(p->usedvaluesfile, p->valueshdu,
+ GAL_TYPE_FLOAT32,
+ p->cp.minmapsize);
+
+ /* Make sure it has the correct size. */
+ if( gal_data_dsize_is_different(p->objects, p->values) )
+ error(EXIT_FAILURE, 0, "`%s' (hdu: %s) and `%s' (hdu: %s) have a"
+ "different dimension/size", p->usedvaluesfile, p->valueshdu,
+ p->objectsfile, p->cp.hdu);
+
+ /* Initially, `p->hasblank' was set based on the objects image, but
+ it may happen that the objects image only has zero values for
+ blank pixels, so we'll also do a check on the input image. */
+ p->hasblank = gal_blank_present(p->objects, 1);
+
+ /* Reset the units of the value-based columns if the input dataset
+ has defined units. */
+ if(p->values->unit)
+ {
+ for(column=p->objectcols; column!=NULL; column=column->next)
+ if( !strcmp(column->unit, MKCATALOG_NO_UNIT) )
+ { free(column->unit); column->unit=p->values->unit; }
+ for(column=p->clumpcols; column!=NULL; column=column->next)
+ if( !strcmp(column->unit, MKCATALOG_NO_UNIT) )
+ { free(column->unit); column->unit=p->values->unit; }
+ }
}
- /* Clean up. */
- keys[0].name=keys[1].name=NULL;
- keys[0].array=keys[1].array=NULL;
- gal_data_array_free(keys, 2, 1);
-}
+ /* Read the Sky image and check its size. */
+ if(p->subtractsky || need_sky)
+ {
+ /* If it wasn't a number, read the dataset into memory. */
+ if(p->sky==NULL)
+ {
+ /* Make sure the HDU is also given. */
+ if(p->skyhdu==NULL)
+ error(EXIT_FAILURE, 0, "%s: no HDU/extension provided for the "
+ "SKY dataset. Atleast one column needs this dataset, or "
+ "you have asked to subtract the Sky from the values.\n\n"
+ "Please use the `--skyhdu' option to give a specific HDU "
+ "using its number (counting from zero) or name. If the "
+ "dataset is in another file, please use `--skyin' to "
+ "give the filename", p->usedskyfile);
+
+ /* Read the Sky dataset. */
+ p->sky=gal_array_read_one_ch_to_type(p->usedskyfile, p->skyhdu,
+ GAL_TYPE_FLOAT32,
+ p->cp.minmapsize);
+
+ /* Check its size and prepare tile structure. */
+ ui_preparation_check_size_read_tiles(p, p->sky, p->usedskyfile,
+ p->skyhdu);
+ }
+ /* Subtract the Sky value. */
+ if(p->subtractsky) ui_subtract_sky(p);
+ }
+ /* Read the Sky standard deviation dataset (if it wasn't already given as
+ a number) and check its size. */
+ if(need_std && p->std==NULL)
+ {
+ /* Make sure the HDU is also given. */
+ if(p->stdhdu==NULL)
+ error(EXIT_FAILURE, 0, "%s: no HDU/extension provided for the "
+ "SKY STANDARD DEVIATION dataset.\n\n"
+ "Atleast one column needs this dataset. Please use the "
+ "`--stdhdu' option to give a specific HDU using its number "
+ "(counting from zero) or name. If the dataset is in another "
+ "file, please use `--stdin' to give the filename",
+ p->usedstdfile);
+
+ /* Read the Sky standard deviation image into memory. */
+ p->std=gal_array_read_one_ch_to_type(p->usedstdfile, p->stdhdu,
+ GAL_TYPE_FLOAT32,
+ p->cp.minmapsize);
+
+ /* Check its size and prepare tile structure. */
+ ui_preparation_check_size_read_tiles(p, p->std, p->usedstdfile,
+ p->stdhdu);
+ }
-/* To make the catalog processing more scalable (and later allow for
- over-lappping regions), we will define a tile for each object. */
-void
-ui_one_tile_per_object(struct mkcatalogparams *p)
-{
- size_t ndim=p->input->ndim;
- int32_t *l, *lf, *start;
- size_t i, d, *min, *max, width=2*ndim;
- size_t *minmax=gal_data_malloc_array(GAL_TYPE_SIZE_T,
- width*p->numobjects, __func__,
- "minmax");
- size_t *coord=gal_data_malloc_array(GAL_TYPE_SIZE_T, ndim, __func__,
- "coord");
+ /* Sanity checks on upper-limit measurements. */
+ if(p->upperlimit)
+ {
+ /* If an upperlimit check was requested, make sure the object number
+ is not larger than the maximum number of labels. */
+ if(p->checkupperlimit[0] != GAL_BLANK_INT32
+ && p->checkupperlimit[0] > p->numobjects)
+ error(EXIT_FAILURE, 0, "%d (object identifer for the "
+ "`--checkupperlimit' option) is larger than the number of "
+ "objects in the input labels (%zu)", p->checkupperlimit[0],
+ p->numobjects);
+
+ /* Read the mask file if it was given. */
+ if(p->upmaskfile)
+ {
+ /* Make sure the HDU for the mask image is given. */
+ if(p->upmaskhdu==NULL)
+ error(EXIT_FAILURE, 0, "%s: no HDU/extension provided, please "
+ "use the `--upmaskhdu' option to specify a specific HDU "
+ "using its number (counting from zero) or name",
+ p->upmaskfile);
+
+ /* Read the mask image. */
+ p->upmask = gal_array_read_one_ch(p->upmaskfile, p->upmaskhdu,
+ p->cp.minmapsize);
+
+ /* Check its size. */
+ if( gal_data_dsize_is_different(p->objects, p->upmask) )
+ error(EXIT_FAILURE, 0, "`%s' (hdu: %s) and `%s' (hdu: %s) have a"
+ "different dimension/size", p->upmaskfile, p->upmaskhdu,
+ p->objectsfile, p->cp.hdu);
+
+ /* If it isn't an integer type, report an error. */
+ if( p->upmask->type==GAL_TYPE_FLOAT32
+ || p->upmask->type==GAL_TYPE_FLOAT64 )
+ error(EXIT_FAILURE, 0, "%s (hdu: %s) has a %s numerical data "
+ "type. Only integer type inputs are acceptable as a mask."
+ "If the values are indeed integers, only placed in a "
+ "floating point container, you can use Gnuastro's "
+ "Arithmetic program to conver the numeric data type",
+ p->upmaskfile, p->upmaskhdu,
+ gal_type_name(p->upmask->type, 1));
+
+ /* Convert the mask to a uint8_t: with a 1 for all non-zero
+ pixels and 0 for zero pixels. */
+ zero=gal_data_alloc(NULL, GAL_TYPE_UINT8, 1, &one, NULL, 1, -1,
+ NULL, NULL, NULL);
+ p->upmask=gal_arithmetic(GAL_ARITHMETIC_OP_NE,
+ ( GAL_ARITHMETIC_INPLACE
+ | GAL_ARITHMETIC_FREE
+ | GAL_ARITHMETIC_NUMOK ),
+ p->upmask, zero);
+ }
+ }
+}
- /* Initialize the minimum and maximum position for each tile/object. So,
- we'll initialize the minimum coordinates to the maximum possible
- `size_t' value (in `GAL_BLANK_SIZE_T') and the maximums to zero. */
- for(i=0;i<p->numobjects;++i)
- for(d=0;d<ndim;++d)
- {
- minmax[ i * width + d ] = GAL_BLANK_SIZE_T; /* Minimum. */
- minmax[ i * width + ndim + d ] = 0; /* Maximum. */
- }
- /* Go over the objects label image and correct the minimum and maximum
- coordinates. */
- start=p->objects->array;
- lf=(l=p->objects->array)+p->objects->size;
- do
- if(*l>0)
- {
- /* Get the coordinates of this pixel. */
- gal_dimension_index_to_coord(l-start, ndim, p->objects->dsize, coord);
- /* Check to see this coordinate is the smallest/largest found so
- far for this label. Note that labels start from 1, while indexs
- here start from zero. */
- min = &minmax[ (*l-1) * width ];
- max = &minmax[ (*l-1) * width + ndim ];
- for(d=0;d<ndim;++d)
- {
- if( coord[d] < min[d] ) min[d] = coord[d];
- if( coord[d] > max[d] ) max[d] = coord[d];
- }
- }
- while(++l<lf);
- /* For a check.
- for(i=0;i<p->numobjects;++i)
- printf("%zu: (%zu, %zu, %zu) --> (%zu, %zu, %zu)\n", i+1, minmax[i*width],
- minmax[i*width+1], minmax[i*width+2], minmax[i*width+3],
- minmax[i*width+4], minmax[i*width+5]);
- exit(0);
- */
- /* Make the tiles. */
- p->tiles=gal_tile_series_from_minmax(p->input, minmax, p->numobjects);
+/* The necessary keywords from the objects or clumps image were read when
+ we were reading them. They were necessary during the
+ pre-processing. Here, we'll read the image from */
+static void
+ui_preparations_read_keywords(struct mkcatalogparams *p)
+{
+ float minstd;
+ gal_data_t *tmp;
+ gal_data_t *keys=NULL;
- /* Clean up. */
- free(coord);
- free(minmax);
+ /* When a Sky standard deviation dataset (not number) is given. */
+ if(p->std && p->std->size>1)
+ {
+ /* Read the keywords from the standard deviation image. */
+ keys=gal_data_array_calloc(2);
+ keys[0].next=&keys[1];
+ keys[0].name="MINSTD"; keys[1].name="MEDSTD";
+ keys[0].type=GAL_TYPE_FLOAT32; keys[1].type=GAL_TYPE_FLOAT32;
+ keys[0].array=&minstd; keys[1].array=&p->medstd;
+ gal_fits_key_read(p->usedstdfile, p->stdhdu, keys, 0, 0);
+
+ /* If the two keywords couldn't be read. We don't want to slow down
+ the user for the median (which needs sorting). So we'll just
+ calculate the minimum which is necessary for the `p->cpscorr'. */
+ if(keys[1].status) p->medstd=NAN;
+ if(keys[0].status)
+ {
+ /* Calculate the minimum STD. */
+ tmp=gal_statistics_minimum(p->std);
+ minstd=*((float *)(tmp->array));
+ gal_data_free(tmp);
+
+ /* If the units are in variance, then take the square root. */
+ if(p->variance) minstd=sqrt(minstd);
+ }
+ p->cpscorr = minstd>1 ? 1.0f : minstd;
+
+ /* Clean up. */
+ keys[0].name=keys[1].name=NULL;
+ keys[0].array=keys[1].array=NULL;
+ gal_data_array_free(keys, 2, 1);
+ }
}
@@ -746,10 +1135,10 @@ ui_preparations_both_names(struct mkcatalogparams *p)
}
else
{
- /* Note that suffix is not used in the text table outputs, so it
+ /* Note that the suffix is not used in the text table outputs, so it
doesn't matter if the output table is not FITS. */
- suffix="_catalog.fits";
- basename = p->inputname;
+ suffix="_cat.fits";
+ basename = p->objectsfile;
}
@@ -767,14 +1156,8 @@ ui_preparations_both_names(struct mkcatalogparams *p)
p->clumpsout=p->objectsout;
}
- /* Revert `keepinputdir' to what it was and free `p->cp.output', we will
- be using `p->objectsout' and `p->clumpsout' from now on. */
- if(p->cp.output)
- {
- p->cp.keepinputdir=keepinputdir;
- free(p->cp.output);
- p->cp.output=NULL;
- }
+ /* Revert `keepinputdir' to what it was. */
+ p->cp.keepinputdir=keepinputdir;
}
@@ -787,7 +1170,7 @@ ui_preparations_outnames(struct mkcatalogparams *p)
{
char *suffix;
- /* Set the output filename */
+ /* The process differs if an output filename has been given. */
if(p->cp.output)
{
/* If the output name is a FITS file, then
@@ -801,13 +1184,12 @@ ui_preparations_outnames(struct mkcatalogparams *p)
else
p->cp.tableformat=GAL_TABLE_FORMAT_TXT;
- /* If a clumps image has been read, then we have two outputs. */
+ /* If a clumps image is present, then we have two outputs. */
if(p->clumps) ui_preparations_both_names(p);
else
{
gal_checkset_writable_remove(p->cp.output, 0, p->cp.dontdelete);
- p->objectsout=p->cp.output;
- p->cp.output=NULL;
+ gal_checkset_allocate_copy(p->cp.output, &p->objectsout);
}
}
else
@@ -819,11 +1201,95 @@ ui_preparations_outnames(struct mkcatalogparams *p)
else
{
suffix = ( p->cp.tableformat==GAL_TABLE_FORMAT_TXT
- ? "._cat.txt" : "_cat.fits" );
- p->objectsout=gal_checkset_automatic_output(&p->cp, p->inputname,
+ ? "_cat.txt" : "_cat.fits" );
+ p->objectsout=gal_checkset_automatic_output(&p->cp, p->objectsfile,
suffix);
}
}
+
+ /* If an upperlimit check image is requsted, then set its filename. */
+ if(p->checkupperlimit)
+ {
+ suffix = ( p->cp.tableformat==GAL_TABLE_FORMAT_TXT
+ ? "_upcheck.txt" : "_upcheck.fits" );
+ p->upcheckout=gal_checkset_automatic_output(&p->cp,
+ ( p->cp.output
+ ? p->cp.output
+ : p->objectsfile),
+ suffix);
+ }
+
+ /* Just to avoid bugs (`p->cp.output' must no longer be used), we'll free
+ it and set it to NULL.*/
+ free(p->cp.output);
+ p->cp.output=NULL;
+}
+
+
+
+
+
+/* To make the catalog processing more scalable (and later allow for
+ over-lappping regions), we will define a tile for each object. */
+void
+ui_one_tile_per_object(struct mkcatalogparams *p)
+{
+ size_t ndim=p->objects->ndim;
+
+ int32_t *l, *lf, *start;
+ size_t i, d, *min, *max, width=2*ndim;
+ size_t *minmax=gal_data_malloc_array(GAL_TYPE_SIZE_T,
+ width*p->numobjects, __func__,
+ "minmax");
+ size_t *coord=gal_data_malloc_array(GAL_TYPE_SIZE_T, ndim, __func__,
+ "coord");
+
+
+ /* Initialize the minimum and maximum position for each tile/object. So,
+ we'll initialize the minimum coordinates to the maximum possible
+ `size_t' value (in `GAL_BLANK_SIZE_T') and the maximums to zero. */
+ for(i=0;i<p->numobjects;++i)
+ for(d=0;d<ndim;++d)
+ {
+ minmax[ i * width + d ] = GAL_BLANK_SIZE_T; /* Minimum. */
+ minmax[ i * width + ndim + d ] = 0; /* Maximum. */
+ }
+
+ /* Go over the objects label image and correct the minimum and maximum
+ coordinates. */
+ start=p->objects->array;
+ lf=(l=p->objects->array)+p->objects->size;
+ do
+ if(*l>0)
+ {
+ /* Get the coordinates of this pixel. */
+ gal_dimension_index_to_coord(l-start, ndim, p->objects->dsize, coord);
+
+ /* Check to see this coordinate is the smallest/largest found so
+ far for this label. Note that labels start from 1, while indexs
+ here start from zero. */
+ min = &minmax[ (*l-1) * width ];
+ max = &minmax[ (*l-1) * width + ndim ];
+ for(d=0;d<ndim;++d)
+ {
+ if( coord[d] < min[d] ) min[d] = coord[d];
+ if( coord[d] > max[d] ) max[d] = coord[d];
+ }
+ }
+ while(++l<lf);
+
+ /* For a check.
+ for(i=0;i<p->numobjects;++i)
+ printf("%zu: (%zu, %zu) --> (%zu, %zu)\n", i+1, minmax[i*width],
+ minmax[i*width+1], minmax[i*width+2], minmax[i*width+3]);
+ */
+
+ /* Make the tiles. */
+ p->tiles=gal_tile_series_from_minmax(p->objects, minmax, p->numobjects);
+
+ /* Clean up. */
+ free(coord);
+ free(minmax);
}
@@ -841,9 +1307,9 @@ ui_preparations_upperlimit(struct mkcatalogparams *p)
if(p->uprange)
{
for(i=0;p->uprange[i]!=-1;++i) ++c;
- if(c!=p->input->ndim)
+ if(c!=p->objects->ndim)
error(EXIT_FAILURE, 0, "%zu values given to `--uprange', but input "
- "has %zu dimensions", c, p->input->ndim);
+ "has %zu dimensions", c, p->objects->ndim);
}
/* Check the number of random samples. */
@@ -897,6 +1363,18 @@ ui_preparations(struct mkcatalogparams *p)
"`--help' for the full list of columns you can ask for");
+ /* Set the actual filenames to use. */
+ ui_set_filenames(p);
+
+
+ /* Read the main input (the objects image). */
+ ui_read_labels(p);
+
+
+ /* Prepare the output columns. */
+ columns_define_alloc(p);
+
+
/* Read the inputs. */
ui_preparations_read_inputs(p);
@@ -910,10 +1388,6 @@ ui_preparations(struct mkcatalogparams *p)
ui_preparations_outnames(p);
- /* Allocate the output columns to fill up with the program. */
- columns_define_alloc(p);
-
-
/* Make the tiles that cover each object. */
ui_one_tile_per_object(p);
@@ -926,6 +1400,24 @@ ui_preparations(struct mkcatalogparams *p)
if( p->hasmag && isnan(p->zeropoint) )
error(EXIT_FAILURE, 0, "no zeropoint specified");
+
+
+ /* Prepare the two internal arrays necessary to sort the clumps catalog
+ by object and clump IDs. We are allocating and filling these in
+ separately (and not using the actual output columns that have the same
+ values), because playing with the output columns can cause bad
+ bugs. If the user wants performance, they are encouraged to run
+ MakeCatalog with `--noclumpsort' and avoid the whole process all
+ together. */
+ if(p->clumps && !p->noclumpsort && p->cp.numthreads>1)
+ {
+ p->hostobjid_c=gal_data_malloc_array(GAL_TYPE_SIZE_T,
+ p->clumpcols->size, __func__,
+ "p->hostobjid_c");
+ p->numclumps_c=gal_data_malloc_array(GAL_TYPE_SIZE_T,
+ p->objectcols->size, __func__,
+ "p->numclumps_c");
+ }
}
@@ -953,8 +1445,8 @@ ui_preparations(struct mkcatalogparams *p)
void
ui_read_check_inputs_setup(int argc, char *argv[], struct mkcatalogparams *p)
{
+ char *tmp;
struct gal_options_common_params *cp=&p->cp;
- char *skyfile, *stdfile, *clumpsfile, *objectsfile;
/* Include the parameters necessary for argp from this program (`args.h')
@@ -983,9 +1475,9 @@ ui_read_check_inputs_setup(int argc, char *argv[], struct
mkcatalogparams *p)
/* Read the options into the program's structure, and check them and
- their relations prior to printing.
+ their relations prior to printing. */
ui_read_check_only_options(p);
- */
+
/* Print the option values if asked. Note that this needs to be done
after the option checks so un-sane values are not printed in the
@@ -1006,22 +1498,36 @@ ui_read_check_inputs_setup(int argc, char *argv[],
struct mkcatalogparams *p)
/* Inform the user. */
if(!p->cp.quiet)
{
- /* Set the names for easy reading. */
- skyfile = p->skyfile ? p->skyfile : p->inputname;
- stdfile = p->stdfile ? p->stdfile : p->inputname;
- clumpsfile = p->clumpsfile ? p->clumpsfile : p->inputname;
- objectsfile = p->objectsfile ? p->objectsfile : p->inputname;
-
/* Write the information. */
printf(PROGRAM_NAME" started on %s", ctime(&p->rawtime));
printf(" - Using %zu CPU thread%s\n", p->cp.numthreads,
p->cp.numthreads==1 ? "." : "s.");
- printf(" - Input: %s (hdu: %s)\n", p->inputname, p->cp.hdu);
- printf(" - Objects: %s (hdu: %s)\n", objectsfile, p->objectshdu);
+ printf(" - Objects: %s (hdu: %s)\n", p->objectsfile, p->cp.hdu);
if(p->clumps)
- printf(" - Clumps: %s (hdu: %s)\n", clumpsfile, p->clumpshdu);
- printf(" - Sky: %s (hdu: %s)\n", skyfile, p->skyhdu);
- printf(" - Sky STD: %s (hdu: %s)\n", stdfile, p->stdhdu);
+ printf(" - Clumps: %s (hdu: %s)\n", p->usedclumpsfile,
+ p->clumpshdu);
+ if(p->values)
+ printf(" - Values: %s (hdu: %s)\n", p->usedvaluesfile,
+ p->valueshdu);
+
+ if(p->subtractsky || p->sky)
+ {
+ if(p->sky->size==1)
+ printf(" - Sky: %g\n", *((float *)(p->sky->array)) );
+ else
+ printf(" - Sky: %s (hdu: %s)\n", p->usedskyfile, p->skyhdu);
+ }
+
+ if(p->std)
+ {
+ tmp = p->variance ? "VAR" : "STD";
+ if(p->std->size==1)
+ printf(" - Sky %s: %g\n", tmp, *((float *)(p->std->array)) );
+ else
+ printf(" - Sky %s: %s (hdu: %s)\n", tmp, p->usedstdfile,
+ p->stdhdu);
+ }
+
if(p->upmaskfile)
printf(" - Upper limit magnitude mask: %s (hdu: %s)\n",
p->upmaskfile, p->cp.hdu);
@@ -1071,7 +1577,7 @@ ui_free_report(struct mkcatalogparams *p, struct timeval
*t1)
/* Free the types of the WCS coordinates (for catalog meta-data). */
if(p->ctype)
{
- for(d=0;d<p->input->ndim;++d)
+ for(d=0;d<p->objects->ndim;++d)
free(p->ctype[d]);
free(p->ctype);
}
@@ -1093,17 +1599,40 @@ ui_free_report(struct mkcatalogparams *p, struct
timeval *t1)
free(p->skyfile);
free(p->stdfile);
free(p->clumpshdu);
- free(p->objectshdu);
+ free(p->valueshdu);
free(p->clumpsfile);
- free(p->objectsfile);
+ free(p->valuesfile);
+ free(p->hostobjid_c);
+ free(p->numclumps_c);
gal_data_free(p->sky);
gal_data_free(p->std);
- gal_data_free(p->input);
+ gal_data_free(p->values);
gal_data_free(p->upmask);
gal_data_free(p->clumps);
gal_data_free(p->objects);
+ if(p->upcheckout) free(p->upcheckout);
gal_data_array_free(p->tiles, p->numobjects, 0);
+ /* If the Sky or its STD image were given in tiles, then we defined a
+ tile structure to deal with them. The initialization of the tile
+ structure is checked with its `ndim' element. */
+ if(p->cp.tl.ndim) gal_tile_full_free_contents(&p->cp.tl);
+
+ /* If an upper limit range warning is necessary, print it here. */
+ if(p->uprangewarning)
+ fprintf(stderr, "\nMore on the WARNING-UPPERLIMIT(s) above: "
+ "In order to obtain a good/robust random distribution (and "
+ "thus a reliable upper-limit measurement), it is necessary "
+ "to have a sufficienty wide enough range (in each dimension). "
+ "As mentioned in the warning(s) above, the available "
+ "range for random sampling of some of the labels in this "
+ "input is less than double their length. If the input is taken "
+ "from a larger dataset, this issue can be solved by using a "
+ "larger part of it. You can also run MakeCatalog with "
+ "`--checkupperlimit' to see the distribution for a special "
+ "object or clump as a table and personally inspect its "
+ "reliability. \n\n");
+
/* Print the final message. */
if(!p->cp.quiet)
gal_timing_report(t1, PROGRAM_NAME" finished in: ", 0);
diff --git a/bin/mkcatalog/ui.h b/bin/mkcatalog/ui.h
index 1d8d28d..b711271 100644
--- a/bin/mkcatalog/ui.h
+++ b/bin/mkcatalog/ui.h
@@ -47,18 +47,17 @@ enum program_args_groups
/* Available letters for short options:
- f g k l u v w x y
- H J L W X Y
+ f g k w x y z
+ E H J L O R W X Y
*/
enum option_keys_enum
{
/* With short-option version. */
- UI_KEY_OBJECTSFILE = 'O', /* General settings. */
- UI_KEY_CLUMPSFILE = 'C',
- UI_KEY_SKYFILE = 's',
- UI_KEY_STDFILE = 't',
- UI_KEY_SKYSUBTRACTED = 'E',
- UI_KEY_THRESHOLD = 'R',
+ UI_KEY_CLUMPSCAT = 'C', /* General settings. */
+ UI_KEY_VALUESFILE = 'v',
+ UI_KEY_CLUMPSFILE = 'l',
+ UI_KEY_INSKY = 's',
+ UI_KEY_INSTD = 't',
UI_KEY_ENVSEED = 'e',
UI_KEY_IDS = 'i', /* Catalog columns. */
@@ -82,11 +81,14 @@ enum option_keys_enum
/* Only with long version (start with a value 1000, the rest will be set
automatically). */
- UI_KEY_OBJECTSHDU = 1000, /* General settings. */
+ UI_KEY_VALUESHDU = 1000, /* General settings. */
UI_KEY_CLUMPSHDU,
UI_KEY_SKYHDU,
UI_KEY_STDHDU,
+ UI_KEY_WITHCLUMPS,
UI_KEY_ZEROPOINT,
+ UI_KEY_VARIANCE,
+ UI_KEY_SUBTRACTSKY,
UI_KEY_SFMAGNSIGMA,
UI_KEY_SFMAGAREA,
UI_KEY_UPMASKFILE,
@@ -95,11 +97,14 @@ enum option_keys_enum
UI_KEY_UPRANGE,
UI_KEY_UPSIGMACLIP,
UI_KEY_UPNSIGMA,
+ UI_KEY_CHECKUPPERLIMIT,
+ UI_KEY_NOCLUMPSORT,
UI_KEY_OBJID, /* Catalog columns. */
UI_KEY_IDINHOSTOBJ,
UI_KEY_CLUMPSAREA,
UI_KEY_WEIGHTAREA,
+ UI_KEY_GEOAREA,
UI_KEY_GEOX,
UI_KEY_GEOY,
UI_KEY_GEOZ,
@@ -121,6 +126,7 @@ enum option_keys_enum
UI_KEY_CLUMPSGEOW1,
UI_KEY_CLUMPSGEOW2,
UI_KEY_CLUMPSGEOW3,
+ UI_KEY_BRIGHTNESSERR,
UI_KEY_CLUMPSBRIGHTNESS,
UI_KEY_NORIVERBRIGHTNESS,
UI_KEY_MEAN,
@@ -130,6 +136,7 @@ enum option_keys_enum
UI_KEY_UPPERLIMITONESIGMA,
UI_KEY_UPPERLIMITSIGMA,
UI_KEY_UPPERLIMITQUANTILE,
+ UI_KEY_UPPERLIMITSKEW,
UI_KEY_RIVERAVE,
UI_KEY_RIVERNUM,
UI_KEY_SKY,
diff --git a/bin/mkcatalog/upperlimit.c b/bin/mkcatalog/upperlimit.c
index d7837b7..00cc09c 100644
--- a/bin/mkcatalog/upperlimit.c
+++ b/bin/mkcatalog/upperlimit.c
@@ -27,6 +27,7 @@ along with Gnuastro. If not, see
<http://www.gnu.org/licenses/>.
#include <error.h>
#include <float.h>
#include <stdlib.h>
+#include <inttypes.h>
#include <gnuastro/tile.h>
#include <gnuastro/threads.h>
@@ -46,14 +47,13 @@ along with Gnuastro. If not, see
<http://www.gnu.org/licenses/>.
static gal_data_t *
upperlimit_make_clump_tiles(struct mkcatalog_passparams *pp)
{
- gal_data_t *input=pp->p->input;
- size_t ndim=input->ndim, *tsize=pp->tile->dsize;
+ gal_data_t *objects=pp->p->objects;
+ size_t ndim=objects->ndim, *tsize=pp->tile->dsize;
- int32_t *O, *C;
gal_data_t *tiles=NULL;
- float *I, *II, *start=input->array;
size_t increment=0, num_increment=1;
size_t i, d, *min, *max, width=2*ndim;
+ int32_t *O, *OO, *C, *start=objects->array;
size_t *coord=gal_data_malloc_array(GAL_TYPE_SIZE_T, ndim, __func__,
"coord");
size_t *minmax=gal_data_malloc_array(GAL_TYPE_SIZE_T,
@@ -75,19 +75,17 @@ upperlimit_make_clump_tiles(struct mkcatalog_passparams *pp)
while( pp->start_end_inc[0] + increment <= pp->start_end_inc[1] )
{
/* Set the pointers for this tile. */
- I = pp->st_i + increment;
- O = pp->st_o + increment;
- C = pp->st_c + increment;
+ C = pp->st_c + increment;
+ OO = ( O = pp->st_o + increment ) + tsize[ndim-1];
/* Go over the contiguous region. */
- II = I + tsize[ndim-1];
do
{
/* Only consider clumps. */
if( *O==pp->object && *C>0 )
{
/* Get the coordinates of this pixel. */
- gal_dimension_index_to_coord(I-start, ndim, input->dsize,
+ gal_dimension_index_to_coord(O-start, ndim, objects->dsize,
coord);
/* Check to see if this coordinate is the smallest/largest
@@ -103,12 +101,12 @@ upperlimit_make_clump_tiles(struct mkcatalog_passparams
*pp)
}
/* Increment the other pointers. */
- ++O; ++C;
+ ++C;
}
- while(++I<II);
+ while(++O<OO);
/* Increment to the next contiguous region. */
- increment += ( gal_tile_block_increment(input, tsize, num_increment++,
+ increment += ( gal_tile_block_increment(objects, tsize, num_increment++,
NULL) );
}
@@ -119,7 +117,7 @@ upperlimit_make_clump_tiles(struct mkcatalog_passparams *pp)
*/
/* Make the tiles. */
- tiles=gal_tile_series_from_minmax(input, minmax, pp->clumpsinobj);
+ tiles=gal_tile_series_from_minmax(objects, minmax, pp->clumpsinobj);
/* Cleanup and return. */
free(coord);
@@ -149,19 +147,21 @@ upperlimit_make_clump_tiles(struct mkcatalog_passparams
*pp)
/*********************************************************************/
/******************* For one tile ********************/
/*********************************************************************/
+/* Set the minimum and maximum possible range to place the FIRST pixel of
+ the object/clump tile over the dataset. */
static void
upperlimit_random_range(struct mkcatalog_passparams *pp, gal_data_t *tile,
size_t *min, size_t *max, int32_t clumplab)
{
struct mkcatalogparams *p=pp->p;
size_t d, tstart, minext, maxext, coord[]={0,0};
- size_t ndim=p->input->ndim, *dsize=p->input->dsize;
+ size_t ndim=p->objects->ndim, *dsize=p->objects->dsize;
/* Set the minimum and maximum acceptable value for the range. */
if(p->uprange)
{
tstart=gal_data_ptr_dist(tile->block->array, tile->array,
- p->input->type);
+ p->objects->type);
gal_dimension_index_to_coord(tstart, ndim, dsize, coord);
}
@@ -211,25 +211,27 @@ upperlimit_random_range(struct mkcatalog_passparams *pp,
gal_data_t *tile,
}
else
{
+ /* We are positioning the FIRST pixel of the tile, not the
+ center. So, the minimum possible value is zero, and in order
+ to not push out of the image, the maximum is the
+ `tile->dsize[d]' away from the edge. */
min[d]=0;
max[d]=dsize[d]-tile->dsize[d]-1;
}
- /* A small sanity check. */
+ /* A small warning to the user if the range isn't large enough. */
if( max[d]-min[d] < 2*tile->dsize[d] )
{
+ p->uprangewarning=1;
if(clumplab)
- fprintf(stderr, "WARNING: object %d clump %d: range of random "
- "positions (%zu) along dimension %zu for upper-limit "
- "calculations is smaller than double of its size (%zu) "
- "in this dimension.\n\n", pp->object, clumplab,
- max[d]-min[d], ndim-d, 2*tile->dsize[d]);
+ fprintf(stderr, "WARNING-UPPERLIMIT: object %d clump %d, "
+ "dimension %zu: range (%zu) < 2*size (%zu).\n",
+ pp->object, clumplab, ndim-d, max[d]-min[d],
+ 2*tile->dsize[d]);
else
- fprintf(stderr, "WARNING: object %d: range of random "
- "positions (%zu) along dimension %zu for upper-limit "
- "calculations is smaller than double of its size (%zu) "
- "in this dimension.\n\n", pp->object, max[d]-min[d],
- ndim-d, 2*tile->dsize[d]);
+ fprintf(stderr, "WARNING-UPPERLIMIT: object %d, dimension %zu: "
+ "range (%zu) < 2*size (%zu).\n", pp->object, ndim-d,
+ max[d]-min[d], 2*tile->dsize[d]);
}
}
}
@@ -250,7 +252,7 @@ upperlimit_random_position(struct mkcatalog_passparams *pp,
gal_data_t *tile,
maximum of the particular generator. It may happen that the labeled
region extends the full range of a dimension. In that case, the only
possible starting point would be 0. */
- if( (int)(p->input->dsize[dim]) - (int)(tile->dsize[dim]) > 0 )
+ if( (int)(p->objects->dsize[dim]) - (int)(tile->dsize[dim]) > 0 )
{
r=gsl_rng_get(pp->rng); /* For easy reading. */
return lrint( (float)(min[dim])
@@ -265,6 +267,197 @@ upperlimit_random_position(struct mkcatalog_passparams
*pp, gal_data_t *tile,
+/* It is necessary to write the upperlimit parameters into the output
+ tables. The same set of information will thus be necessary both in the
+ upperlimit check table and also the final output. This function will do
+ the job in both cases.
+
+ Note that in the check output, the sigma-clipping information is not
+ used/necessary, so to avoid confusion, we won't write it.
+*/
+void
+upperlimit_write_comments(struct mkcatalogparams *p,
+ gal_list_str_t **comments, int withsigclip)
+{
+ char *str;
+
+ if(p->cp.tableformat==GAL_TABLE_FORMAT_TXT)
+ {
+ if(asprintf(&str, "--------- Upper-limit measurement ---------")<0)
+ error(EXIT_FAILURE, 0, "%s: asprintf allocation", __func__);
+ gal_list_str_add(comments, str, 0);
+ }
+
+ if( asprintf(&str, "Number of usable random samples: %zu", p->upnum)<0 )
+ error(EXIT_FAILURE, 0, "%s: asprintf allocation", __func__);
+ gal_list_str_add(comments, str, 0);
+
+ if(p->uprange)
+ {
+ switch(p->objects->ndim)
+ {
+ case 2:
+ if( asprintf(&str, "Range of random samples about target: "
+ "%zu, %zu", p->uprange[1], p->uprange[0])<0 )
+ error(EXIT_FAILURE, 0, "%s: asprintf allocation", __func__);
+ break;
+ case 3:
+ if( asprintf(&str, "Range of random samples about target: %zu, "
+ "%zu, %zu", p->uprange[2], p->uprange[1],
+ p->uprange[0])<0 )
+ error(EXIT_FAILURE, 0, "%s: asprintf allocation", __func__);
+ break;
+ default:
+ error(EXIT_FAILURE, 0, "%s: a bug! Please contact us at %s to "
+ "address the problem. The value %zu is not recognized for "
+ "`p->input->ndim'", __func__, PACKAGE_BUGREPORT,
+ p->objects->ndim);
+ }
+ gal_list_str_add(comments, str, 0);
+ }
+
+ if( asprintf(&str, "Random number generator name: %s", p->rngname)<0 )
+ error(EXIT_FAILURE, 0, "%s: asprintf allocation", __func__);
+ gal_list_str_add(comments, str, 0);
+
+ if( asprintf(&str, "Random number generator seed: %"PRIu64, p->seed)<0 )
+ error(EXIT_FAILURE, 0, "%s: asprintf allocation", __func__);
+ gal_list_str_add(comments, str, 0);
+
+ if(withsigclip)
+ {
+ if( asprintf(&str, "Multiple of STD used for sigma-clipping: %.3f",
+ p->upsigmaclip[0])<0 )
+ error(EXIT_FAILURE, 0, "%s: asprintf allocation", __func__);
+ gal_list_str_add(comments, str, 0);
+
+ if(p->upsigmaclip[1]>=1.0f)
+ {
+ if( asprintf(&str, "Number of clips for sigma-clipping: %.0f",
+ p->upsigmaclip[1])<0 )
+ error(EXIT_FAILURE, 0, "%s: asprintf allocation", __func__);
+ }
+ else
+ {
+ if( asprintf(&str, "Tolerance level to sigma-clipping: %.3f",
+ p->upsigmaclip[1])<0 )
+ error(EXIT_FAILURE, 0, "%s: asprintf allocation", __func__);
+ }
+ gal_list_str_add(comments, str, 0);
+
+ if( p->oiflag[ OCOL_UPPERLIMIT_B ] )
+ {
+ if( asprintf(&str, "Multiple of sigma-clipped STD for upper-limit: "
+ "%.3f", p->upnsigma)<0 )
+ error(EXIT_FAILURE, 0, "%s: asprintf allocation", __func__);
+ gal_list_str_add(comments, str, 0);
+ }
+ }
+}
+
+
+
+
+
+/* Write the values into a table for the user */
+static void
+upperlimit_write_check(struct mkcatalogparams *p, gal_list_sizet_t *check_x,
+ gal_list_sizet_t *check_y, gal_list_sizet_t *check_z,
+ gal_list_f32_t *check_s)
+{
+ float *sarr;
+ gal_data_t *x, *y, *z, *s;
+ char *tmp=NULL, *tmp2=NULL;
+ gal_list_str_t *comments=NULL;
+ size_t *xarr, *yarr, *zarr=NULL, tnum, ttnum, num;
+
+
+ /* Convert the lists to an array. */
+ xarr=gal_list_sizet_to_array(check_x, 1, &num);
+ yarr=gal_list_sizet_to_array(check_y, 1, &tnum);
+ if(check_z) zarr=gal_list_sizet_to_array(check_z, 1, &ttnum);
+ if(tnum!=num || (check_z && ttnum!=num) )
+ error(EXIT_FAILURE, 0, "%s: a bug! Please contact us at %s to fix the "
+ "problem. For some reason the size of the input lists don't "
+ "match (%zu, %zu)", __func__, PACKAGE_BUGREPORT, tnum, num);
+ sarr=gal_list_f32_to_array(check_s, 1, &tnum);
+ if(tnum!=num)
+ error(EXIT_FAILURE, 0, "%s: a bug! Please contact us at %s to fix the "
+ "problem. For some reason the size of the input lists don't "
+ "match (%zu, %zu)", __func__, PACKAGE_BUGREPORT, tnum, num);
+
+
+ /* Put the arrays into a data container. */
+ x=gal_data_alloc(xarr, GAL_TYPE_SIZE_T, 1, &num, NULL, 0, p->cp.minmapsize,
+ "RANDOM_X", "pixel",
+ "X-axis position of random footprint's first pixel.");
+ y=gal_data_alloc(yarr, GAL_TYPE_SIZE_T, 1, &num, NULL, 0, p->cp.minmapsize,
+ "RANDOM_Y", "pixel",
+ "Y-axis position of random footprint's first pixel.");
+ if(check_z)
+ z=gal_data_alloc(zarr, GAL_TYPE_SIZE_T, 1, &num, NULL, 0,
+ p->cp.minmapsize, "RANDOM_Z", "pixel",
+ "Z-axis position of random footprint's first pixel.");
+ s=gal_data_alloc(sarr, GAL_TYPE_FLOAT32, 1, &num, NULL, 0, p->cp.minmapsize,
+ "RANDOM_SUM",
+ p->values->unit ? p->values->unit : "input-units",
+ "Sum of pixel values over random footprint.");
+
+
+ /* Convert the unsigned 64-bit values to 32-bit because the FITS table
+ format doesn't recognize 64-bit integers.*/
+ x=gal_data_copy_to_new_type_free(x, GAL_TYPE_UINT32);
+ y=gal_data_copy_to_new_type_free(y, GAL_TYPE_UINT32);
+ if(check_z) z=gal_data_copy_to_new_type_free(y, GAL_TYPE_UINT32);
+
+
+ /* Write exactly what object/clump this table is for. */
+ if( p->checkupperlimit[1]!=GAL_BLANK_INT32 )
+ if( asprintf(&tmp2, ", Clump %d", p->checkupperlimit[1]) <0 )
+ error(EXIT_FAILURE, 0, "%s: asprintf allocation", __func__);
+ if( asprintf(&tmp, "Upperlimit distribution for Object %d%s",
+ p->checkupperlimit[0],
+ ( p->checkupperlimit[1]==GAL_BLANK_INT32
+ ? "" : tmp2) ) <0 )
+ error(EXIT_FAILURE, 0, "%s: asprintf allocation", __func__);
+ gal_list_str_add(&comments, tmp, 0);
+ if(tmp2) {free(tmp2); tmp2=NULL;}
+
+
+ /* Write the basic info, and conclude the comments. */
+ mkcatalog_write_inputs_in_comments(p, &comments, 0, 0);
+ upperlimit_write_comments(p, &comments, 0);
+ if(p->cp.tableformat==GAL_TABLE_FORMAT_TXT)
+ {
+ if( asprintf(&tmp, "--------- Table columns ---------")<0 )
+ error(EXIT_FAILURE, 0, "%s: asprintf allocation", __func__);
+ gal_list_str_add(&comments, tmp, 0);
+ }
+
+
+ /* Define a list from the containers and write them into a table. */
+ x->next=y;
+ if(check_z) { y->next=z; z->next=s; }
+ else { y->next=s; }
+ gal_list_str_reverse(&comments);
+ gal_table_write(x, comments, p->cp.tableformat, p->upcheckout,
+ "UPPERLIMIT_CHECK");
+
+ /* Inform the user. */
+ if(!p->cp.quiet)
+ printf(" - Upperlimit check table: %s\n", p->upcheckout);
+
+ /* Clean up. */
+ gal_data_free(x);
+ gal_data_free(y);
+ gal_data_free(s);
+ if(check_z) gal_data_free(z);
+}
+
+
+
+
+
/* Given the distribution of values, do the upper-limit calculations. */
static void
upperlimit_measure(struct mkcatalog_passparams *pp, int32_t clumplab,
@@ -309,14 +502,18 @@ upperlimit_measure(struct mkcatalog_passparams *pp,
int32_t clumplab,
pp->up_vals->size=pp->up_vals->dsize[0]=init_size;
scarr=sigclip->array;
- /* Write the raw sigma. */
+ /* 1-sigma. */
col = clumplab ? CCOL_UPPERLIMIT_S : OCOL_UPPERLIMIT_S;
o[col] = scarr[3];
- /* Write the multiple of `upnsigma'. */
+ /* sigma multiplied by `upnsigma'. */
col = clumplab ? CCOL_UPPERLIMIT_B : OCOL_UPPERLIMIT_B;
o[col] = scarr[3] * p->upnsigma;
+ /* Nonparametric skewness [ (Mean-Median)/STD ]. */
+ col = clumplab?CCOL_UPPERLIMIT_SKEW:OCOL_UPPERLIMIT_SKEW;
+ o[col] = ( scarr[2] - scarr[1] ) / scarr[3];
+
/* Clean up. */
gal_data_free(sigclip);
}
@@ -351,8 +548,8 @@ upperlimit_measure(struct mkcatalog_passparams *pp, int32_t
clumplab,
}
else
{
- o[ clumplab ? CCOL_UPPERLIMIT_S : OCOL_UPPERLIMIT_S ] = NAN;
o[ clumplab ? CCOL_UPPERLIMIT_B : OCOL_UPPERLIMIT_B ] = NAN;
+ o[ clumplab ? CCOL_UPPERLIMIT_S : OCOL_UPPERLIMIT_S ] = NAN;
o[ clumplab ? CCOL_UPPERLIMIT_Q : OCOL_UPPERLIMIT_Q ] = NAN;
}
}
@@ -366,21 +563,37 @@ upperlimit_one_tile(struct mkcatalog_passparams *pp,
gal_data_t *tile,
unsigned long seed, int32_t clumplab)
{
struct mkcatalogparams *p=pp->p;
- size_t ndim=p->input->ndim, *dsize=p->input->dsize;
+ size_t ndim=p->objects->ndim, *dsize=p->objects->dsize;
double sum;
void *tarray;
- int continueparse;
uint8_t *M=NULL, *st_m=NULL;
- float *uparr=pp->up_vals->array;
- float *I, *II, *SK, *st_i, *st_sky;
+ int continueparse, writecheck=0;
+ struct gal_list_f32_t *check_s=NULL;
+ float *V, *st_v, *uparr=pp->up_vals->array;
size_t d, tcounter=0, counter=0, se_inc[2];
size_t min[3], max[3], increment, num_increment;
- int32_t *O, *oO, *st_o, *st_oo, *st_oc, *oC=NULL;
+ int32_t *O, *OO, *oO, *st_o, *st_oo, *st_oc, *oC=NULL;
size_t maxcount = p->upnum * MKCATALOG_UPPERLIMIT_STOP_MULTIP;
+ struct gal_list_sizet_t *check_x=NULL, *check_y=NULL, *check_z=NULL;
size_t *rcoord=gal_data_malloc_array(GAL_TYPE_SIZE_T, ndim, __func__,
"rcoord");
+ /* See if a check table must be created for this distribution. */
+ if( p->checkupperlimit[0]==pp->object )
+ {
+ /* We are on a clump */
+ if( clumplab )
+ {
+ if( p->checkupperlimit[1]==clumplab )
+ writecheck=1;
+ }
+ else
+ if( p->checkupperlimit[1]==GAL_BLANK_INT32 )
+ writecheck=1;
+ }
+
+
/* Initializations. */
tarray=tile->array;
gsl_rng_set(pp->rng, seed);
@@ -406,9 +619,9 @@ upperlimit_one_tile(struct mkcatalog_passparams *pp,
gal_data_t *tile,
rcoord[d] = upperlimit_random_position(pp, tile, d, min, max);
/* Set the tile's new starting pointer. */
- tile->array = gal_data_ptr_increment(p->input->array,
+ tile->array = gal_data_ptr_increment(p->objects->array,
gal_dimension_coord_to_index(ndim, dsize, rcoord),
- p->input->type);
+ p->objects->type);
/* Starting and ending coordinates for this random position, note
that in `pp' we have the starting and ending coordinates of the
@@ -419,65 +632,98 @@ upperlimit_one_tile(struct mkcatalog_passparams *pp,
gal_data_t *tile,
sum = 0.0f;
/* Starting pointers for the random tile. */
- st_i = gal_tile_start_end_ind_inclusive(tile, p->input, se_inc);
+ st_v = gal_tile_start_end_ind_inclusive(tile, p->values, se_inc);
st_o = (int32_t *)(p->objects->array) + se_inc[0];
- st_sky = (float *)(p->sky->array) + se_inc[0];
if(p->upmask) st_m = (uint8_t *)(p->upmask->array) + se_inc[0];
/* Parse over this object/clump. */
while( se_inc[0] + increment <= se_inc[1] )
{
/* Set the pointers. */
- I = st_i + increment; /* Random tile. */
- SK = st_sky + increment; /* Random tile. */
- O = st_o + increment; /* Random tile. */
- if(st_m) M = st_m + increment; /* Random tile. */
- oO = st_oo + increment; /* Original tile. */
- if(clumplab) oC = st_oc + increment; /* Original tile. */
+ V = st_v + increment; /* Random tile. */
+ O = st_o + increment; /* Random tile. */
+ if(st_m) M = st_m + increment; /* Random tile. */
+ oO = st_oo + increment; /* Original tile. */
+ if(clumplab) oC = st_oc + increment; /* Original tile. */
/* Parse over this contiguous region, similar to the first and
second pass functions. */
- II = I + tile->dsize[ndim-1];
+ OO = O + tile->dsize[ndim-1];
do
{
/* Only use pixels over this object/clump. */
- if( *oO==pp->object
- && ( oC==NULL || clumplab==0 || *oC==clumplab ) )
+ if( *oO==pp->object && ( oC==NULL || *oC==clumplab ) )
{
- if( *O || (M && *M) || ( p->hasblank && isnan(*I) ) )
+ /* If this pixel is a non-zero object code, or is masked,
+ or has a blank value, then stop parsing. */
+ if( *O || (M && *M) || ( p->hasblank && isnan(*V) ) )
continueparse=0;
else
- sum += *I-*SK;
+ sum += *V;
}
/* Increment the other pointers. */
- ++SK; ++O; ++oO; if(oC) ++oC;
+ ++V;
+ ++oO;
+ if(M) ++M;
+ if(oC) ++oC;
}
- while(continueparse && ++I<II);
+ while(continueparse && ++O<OO);
/* Increment to the next contiguous region of this tile. */
if(continueparse)
- increment += ( gal_tile_block_increment(p->input, dsize,
+ increment += ( gal_tile_block_increment(p->objects, dsize,
num_increment++, NULL) );
else break;
}
- /* Further processing is only necessary if this random tile
- actually covered the sky region. */
+ /* Further processing is only necessary if this random tile was fully
+ parsed. */
if(continueparse) uparr[ counter++ ] = sum;
+ /* If a check is necessary, write in the values. */
+ if(writecheck)
+ {
+ switch(ndim)
+ {
+ case 2:
+ gal_list_sizet_add(&check_x, rcoord[1]+1);
+ gal_list_sizet_add(&check_y, rcoord[0]+1);
+ break;
+
+ case 3:
+ gal_list_sizet_add(&check_x, rcoord[2]+1);
+ gal_list_sizet_add(&check_y, rcoord[1]+1);
+ gal_list_sizet_add(&check_z, rcoord[0]+1);
+ break;
+
+ default:
+ error(EXIT_FAILURE, 0, "%s: a bug! Please contact us at %s "
+ "to fix the problem. `ndim' value of %zu is not "
+ "recognized", __func__, PACKAGE_BUGREPORT, ndim);
+ }
+ gal_list_f32_add(&check_s, continueparse ? sum : NAN);
+ }
+
/* Increment the total-counter. */
++tcounter;
}
+ /* If a check is necessary, then write the values. */
+ if(writecheck)
+ upperlimit_write_check(p, check_x, check_y, check_z, check_s);
+
/* Do the measurement on the random distribution. */
upperlimit_measure(pp, clumplab, counter==p->upnum);
/* Reset the tile's array pointer, clean up and return. */
- tile->array=tarray;
free(rcoord);
+ tile->array=tarray;
+ gal_list_f32_free(check_s);
+ gal_list_sizet_free(check_x);
+ gal_list_sizet_free(check_y);
}
@@ -518,6 +764,17 @@ upperlimit_calculate(struct mkcatalog_passparams *pp)
within this object. */
if(p->clumps && pp->clumpsinobj)
{
+ /* If an upper-limit check image is requested, then make sure that
+ the clump label is not more than the number of clumps in this
+ object. */
+ if( p->checkupperlimit[0] == pp->object
+ && p->checkupperlimit[1] != GAL_BLANK_INT32
+ && p->checkupperlimit[1] > pp->clumpsinobj )
+ error(EXIT_FAILURE, 0, "object %d has %zu clumps, but an upperlimit "
+ "check table (using the `--checkupperlimit' option) has been "
+ "requested for clump %d", pp->object, pp->clumpsinobj,
+ p->checkupperlimit[1]);
+
/* Make tiles covering the clumps. */
clumptiles=upperlimit_make_clump_tiles(pp);
diff --git a/bin/mkcatalog/upperlimit.h b/bin/mkcatalog/upperlimit.h
index 143252f..001ed83 100644
--- a/bin/mkcatalog/upperlimit.h
+++ b/bin/mkcatalog/upperlimit.h
@@ -24,6 +24,10 @@ along with Gnuastro. If not, see
<http://www.gnu.org/licenses/>.
#define UPPERLIMIT_H
void
+upperlimit_write_comments(struct mkcatalogparams *p,
+ gal_list_str_t **comments, int withsigclip);
+
+void
upperlimit_calculate(struct mkcatalog_passparams *pp);
#endif
diff --git a/bin/mknoise/mknoise.c b/bin/mknoise/mknoise.c
index 95f0a0b..4c1058a 100644
--- a/bin/mknoise/mknoise.c
+++ b/bin/mknoise/mknoise.c
@@ -107,7 +107,8 @@ convertsaveoutput(struct mknoiseparams *p)
void
mknoise(struct mknoiseparams *p)
{
- double *d, *df, background=p->background, instrumental=p->instrumental;
+ double *d, *df, background=p->background;
+ double instpowtwo = p->instrumental*p->instrumental;
/* Add the noise: */
df=(d=p->input->array)+p->input->size;
@@ -122,7 +123,7 @@ mknoise(struct mknoiseparams *p)
do
*d += ( background
+ gsl_ran_gaussian(p->rng,
- sqrt( instrumental + background + *d )) );
+ sqrt( instpowtwo + background + *d )) );
while(++d<df);
}
diff --git a/bin/mknoise/ui.c b/bin/mknoise/ui.c
index eba6e4f..f713784 100644
--- a/bin/mknoise/ui.c
+++ b/bin/mknoise/ui.c
@@ -30,6 +30,7 @@ along with Gnuastro. If not, see
<http://www.gnu.org/licenses/>.
#include <gnuastro/wcs.h>
#include <gnuastro/fits.h>
+#include <gnuastro/array.h>
#include <gnuastro/table.h>
#include <gnuastro-internal/timing.h>
@@ -281,8 +282,9 @@ void
ui_preparations(struct mknoiseparams *p)
{
/* Read the input image as a double type */
- p->input=gal_fits_img_read_to_type(p->inputname, p->cp.hdu,
- GAL_TYPE_FLOAT64, p->cp.minmapsize, 0,0);
+ p->input=gal_array_read_one_ch_to_type(p->inputname, p->cp.hdu,
+ GAL_TYPE_FLOAT64, p->cp.minmapsize);
+ p->input->wcs=gal_wcs_read(p->inputname, p->cp.hdu, 0, 0, &p->input->nwcs);
/* If we are dealing with an input table, make sure the format of the
diff --git a/bin/mkprof/main.h b/bin/mkprof/main.h
index b31e229..e71a125 100644
--- a/bin/mkprof/main.h
+++ b/bin/mkprof/main.h
@@ -128,7 +128,7 @@ struct mkprofparams
size_t *shift; /* Shift along axeses position of profiles. */
uint8_t prepforconv; /* Shift and expand by size of first psf. */
float zeropoint; /* Magnitude of zero point flux. */
- double circumwidth; /* Width of circumference (inward). */
+ float circumwidth; /* Width of circumference (inward). */
uint8_t replace; /* Replace overlaping profile pixel values. */
uint8_t magatpeak; /* Mag only for peak pixel, not all profile.*/
uint8_t envseed; /* Use GSL_RNG_SEED for random seed. */
diff --git a/bin/mkprof/profiles.c b/bin/mkprof/profiles.c
index 13369a0..f350ddd 100644
--- a/bin/mkprof/profiles.c
+++ b/bin/mkprof/profiles.c
@@ -168,7 +168,8 @@ profiles_sersic(struct mkonthread *mkp)
double
profiles_circumference(struct mkonthread *mkp)
{
- return mkp->r > mkp->intruncr ? mkp->fixedvalue : 0.0f;
+ return ( (mkp->r > mkp->intruncr && mkp->r <= mkp->truncr)
+ ? mkp->fixedvalue : 0.0f );
}
diff --git a/bin/mkprof/ui.c b/bin/mkprof/ui.c
index 2f54739..86250c0 100644
--- a/bin/mkprof/ui.c
+++ b/bin/mkprof/ui.c
@@ -31,6 +31,7 @@ along with Gnuastro. If not, see
<http://www.gnu.org/licenses/>.
#include <gnuastro/wcs.h>
#include <gnuastro/box.h>
#include <gnuastro/fits.h>
+#include <gnuastro/array.h>
#include <gnuastro/blank.h>
#include <gnuastro/table.h>
@@ -1325,9 +1326,11 @@ ui_prepare_canvas(struct mkprofparams *p)
else
{
/* Read the image. */
- p->out=gal_fits_img_read_to_type(p->backname, p->backhdu,
- GAL_TYPE_FLOAT32,
- p->cp.minmapsize, 0, 0);
+ p->out=gal_array_read_one_ch_to_type(p->backname, p->backhdu,
+ GAL_TYPE_FLOAT32,
+ p->cp.minmapsize);
+ p->out->wcs=gal_wcs_read(p->backname, p->backhdu, 0, 0,
+ &p->out->nwcs);
/* Put the WCS structure and number of dimensions in the
MakeProfiles's main structure for generality. The WCS
diff --git a/bin/noisechisel/Makefile.am b/bin/noisechisel/Makefile.am
index 70765ea..0fab8ea 100644
--- a/bin/noisechisel/Makefile.am
+++ b/bin/noisechisel/Makefile.am
@@ -30,11 +30,11 @@ bin_PROGRAMS = astnoisechisel
astnoisechisel_LDADD = -lgnuastro
-astnoisechisel_SOURCES = main.c ui.c clumps.c detection.c noisechisel.c \
- sky.c segmentation.c threshold.c
+astnoisechisel_SOURCES = main.c ui.c detection.c noisechisel.c sky.c \
+ threshold.c
-EXTRA_DIST = main.h authors-cite.h args.h ui.h clumps.h detection.h \
- kernel-3d.h noisechisel.h segmentation.h sky.h threshold.h
+EXTRA_DIST = main.h authors-cite.h args.h ui.h detection.h noisechisel.h \
+ sky.h threshold.h
diff --git a/bin/noisechisel/args.h b/bin/noisechisel/args.h
index 77cea84..76b6cda 100644
--- a/bin/noisechisel/args.h
+++ b/bin/noisechisel/args.h
@@ -1,5 +1,5 @@
/*********************************************************************
-NoiseChisel - Detect and segment signal in a noisy dataset.
+NoiseChisel - Detect signal in a noisy dataset.
NoiseChisel is part of GNU Astronomy Utilities (Gnuastro) package.
Original author:
@@ -73,13 +73,13 @@ struct argp_option program_options[] =
GAL_OPTIONS_NOT_SET
},
{
- "convolvedhdu",
- UI_KEY_CONVOLVEDHDU,
+ "chdu",
+ UI_KEY_CHDU,
"STR",
0,
"HDU/extension of convolved image in file.",
GAL_OPTIONS_GROUP_INPUT,
- &p->convolvedhdu,
+ &p->chdu,
GAL_TYPE_STRING,
GAL_OPTIONS_RANGE_ANY,
GAL_OPTIONS_NOT_MANDATORY,
@@ -99,57 +99,20 @@ struct argp_option program_options[] =
GAL_OPTIONS_NOT_SET
},
{
- "wkhdu",
- UI_KEY_WKHDU,
+ "whdu",
+ UI_KEY_WHDU,
"STR",
0,
"HDU containing wide kernel image.",
GAL_OPTIONS_GROUP_INPUT,
- &p->wkhdu,
+ &p->whdu,
GAL_TYPE_STRING,
GAL_OPTIONS_RANGE_ANY,
GAL_OPTIONS_NOT_MANDATORY,
GAL_OPTIONS_NOT_SET
},
- {
- "skysubtracted",
- UI_KEY_SKYSUBTRACTED,
- 0,
- 0,
- "Input is Sky subtracted (for error estimation).",
- GAL_OPTIONS_GROUP_INPUT,
- &p->skysubtracted,
- GAL_OPTIONS_NO_ARG_TYPE,
- GAL_OPTIONS_RANGE_0_OR_1,
- GAL_OPTIONS_NOT_MANDATORY,
- GAL_OPTIONS_NOT_SET
- },
- {
- "minskyfrac",
- UI_KEY_MINSKYFRAC,
- "FLT",
- 0,
- "Min. fraction of undetected area in tile.",
- GAL_OPTIONS_GROUP_INPUT,
- &p->minskyfrac,
- GAL_TYPE_FLOAT32,
- GAL_OPTIONS_RANGE_GE_0_LE_1,
- GAL_OPTIONS_MANDATORY,
- GAL_OPTIONS_NOT_SET
- },
- {
- "minnumfalse",
- UI_KEY_MINNUMFALSE,
- "INT",
- 0,
- "Minimum number for S/N estimation.",
- GAL_OPTIONS_GROUP_INPUT,
- &p->minnumfalse,
- GAL_TYPE_SIZE_T,
- GAL_OPTIONS_RANGE_GT_0,
- GAL_OPTIONS_MANDATORY,
- GAL_OPTIONS_NOT_SET
- },
+
+
/* Tessellation. */
@@ -174,19 +137,31 @@ struct argp_option program_options[] =
/* Output options. */
{
- "onlydetection",
- UI_KEY_ONLYDETECTION,
+ "rawoutput",
+ UI_KEY_RAWOUTPUT,
0,
0,
- "Stop at the end of detection.",
+ "Output only detection labels & 1-elem/tile grid.",
GAL_OPTIONS_GROUP_OUTPUT,
- &p->onlydetection,
+ &p->rawoutput,
+ GAL_OPTIONS_NO_ARG_TYPE,
+ GAL_OPTIONS_RANGE_0_OR_1,
+ GAL_OPTIONS_NOT_MANDATORY,
+ GAL_OPTIONS_NOT_SET
+ },
+ {
+ "label",
+ UI_KEY_LABEL,
+ 0,
+ 0,
+ "Label/count detected pixels that are connected.",
+ GAL_OPTIONS_GROUP_OUTPUT,
+ &p->label,
GAL_OPTIONS_NO_ARG_TYPE,
GAL_OPTIONS_RANGE_0_OR_1,
GAL_OPTIONS_NOT_MANDATORY,
GAL_OPTIONS_NOT_SET
},
-
@@ -355,6 +330,19 @@ struct argp_option program_options[] =
gal_options_read_sigma_clip
},
{
+ "minskyfrac",
+ UI_KEY_MINSKYFRAC,
+ "FLT",
+ 0,
+ "Min. fraction of undetected area in tile.",
+ UI_GROUP_DETECTION,
+ &p->minskyfrac,
+ GAL_TYPE_FLOAT32,
+ GAL_OPTIONS_RANGE_GE_0_LE_1,
+ GAL_OPTIONS_MANDATORY,
+ GAL_OPTIONS_NOT_SET
+ },
+ {
"checkdetsky",
UI_KEY_CHECKDETSKY,
0,
@@ -381,39 +369,52 @@ struct argp_option program_options[] =
GAL_OPTIONS_NOT_SET
},
{
- "detsnminarea",
- UI_KEY_DETSNMINAREA,
+ "snminarea",
+ UI_KEY_SNMINAREA,
"INT",
0,
"Min. pseudo-detection area for S/N dist.",
UI_GROUP_DETECTION,
- &p->detsnminarea,
+ &p->snminarea,
GAL_TYPE_SIZE_T,
GAL_OPTIONS_RANGE_ANY,
GAL_OPTIONS_MANDATORY,
GAL_OPTIONS_NOT_SET
},
{
- "checkdetsn",
- UI_KEY_CHECKDETSN,
+ "checksn",
+ UI_KEY_CHECKSN,
0,
0,
"Save pseudo-detection S/N values to a file.",
UI_GROUP_DETECTION,
- &p->checkdetsn,
+ &p->checksn,
GAL_OPTIONS_NO_ARG_TYPE,
GAL_OPTIONS_RANGE_0_OR_1,
GAL_OPTIONS_NOT_MANDATORY,
GAL_OPTIONS_NOT_SET
},
{
- "detquant",
- UI_KEY_DETQUANT,
+ "minnumfalse",
+ UI_KEY_MINNUMFALSE,
+ "INT",
+ 0,
+ "Minimum number for S/N estimation.",
+ UI_GROUP_DETECTION,
+ &p->minnumfalse,
+ GAL_TYPE_SIZE_T,
+ GAL_OPTIONS_RANGE_GT_0,
+ GAL_OPTIONS_MANDATORY,
+ GAL_OPTIONS_NOT_SET
+ },
+ {
+ "snquant",
+ UI_KEY_SNQUANT,
"FLT",
0,
"Quantile in pseudo-det. to define true.",
UI_GROUP_DETECTION,
- &p->detquant,
+ &p->snquant,
GAL_TYPE_FLOAT32,
GAL_OPTIONS_RANGE_GT_0_LT_1,
GAL_OPTIONS_MANDATORY,
@@ -489,134 +490,6 @@ struct argp_option program_options[] =
- /* Segmentation */
- {
- 0, 0, 0, 0,
- "Segmentation:",
- UI_GROUP_SEGMENTATION
- },
- {
- "segsnminarea",
- UI_KEY_SEGSNMINAREA,
- "INT",
- 0,
- "Minimum area of clumps for S/N estimation.",
- UI_GROUP_SEGMENTATION,
- &p->segsnminarea,
- GAL_TYPE_SIZE_T,
- GAL_OPTIONS_RANGE_GT_0,
- GAL_OPTIONS_MANDATORY,
- GAL_OPTIONS_NOT_SET
- },
- {
- "checkclumpsn",
- UI_KEY_CHECKCLUMPSN,
- 0,
- 0,
- "Save Sky clump S/N values into a file.",
- UI_GROUP_SEGMENTATION,
- &p->checkclumpsn,
- GAL_OPTIONS_NO_ARG_TYPE,
- GAL_OPTIONS_RANGE_0_OR_1,
- GAL_OPTIONS_NOT_MANDATORY,
- GAL_OPTIONS_NOT_SET
- },
- {
- "segquant",
- UI_KEY_SEGQUANT,
- "FLT",
- 0,
- "S/N Quantile of true sky clumps.",
- UI_GROUP_SEGMENTATION,
- &p->segquant,
- GAL_TYPE_FLOAT32,
- GAL_OPTIONS_RANGE_GT_0,
- GAL_OPTIONS_MANDATORY,
- GAL_OPTIONS_NOT_SET
- },
- {
- "keepmaxnearriver",
- UI_KEY_KEEPMAXNEARRIVER,
- 0,
- 0,
- "Keep clumps with peak touching a river.",
- UI_GROUP_SEGMENTATION,
- &p->keepmaxnearriver,
- GAL_OPTIONS_NO_ARG_TYPE,
- GAL_OPTIONS_RANGE_0_OR_1,
- GAL_OPTIONS_NOT_MANDATORY,
- GAL_OPTIONS_NOT_SET
- },
- {
- "gthresh",
- UI_KEY_GTHRESH,
- "FLT",
- 0,
- "Multiple of STD to stop growing clumps.",
- UI_GROUP_SEGMENTATION,
- &p->gthresh,
- GAL_TYPE_FLOAT32,
- GAL_OPTIONS_RANGE_ANY,
- GAL_OPTIONS_MANDATORY,
- GAL_OPTIONS_NOT_SET
- },
- {
- "minriverlength",
- UI_KEY_MINRIVERLENGTH,
- "INT",
- 0,
- "Minimum len of useful grown clump rivers.",
- UI_GROUP_SEGMENTATION,
- &p->minriverlength,
- GAL_TYPE_SIZE_T,
- GAL_OPTIONS_RANGE_ANY,
- GAL_OPTIONS_MANDATORY,
- GAL_OPTIONS_NOT_SET
- },
- {
- "objbordersn",
- UI_KEY_OBJBORDERSN,
- "FLT",
- 0,
- "Min. S/N for grown clumps as one object.",
- UI_GROUP_SEGMENTATION,
- &p->objbordersn,
- GAL_TYPE_FLOAT32,
- GAL_OPTIONS_RANGE_GE_0,
- GAL_OPTIONS_MANDATORY,
- GAL_OPTIONS_NOT_SET
- },
- {
- "grownclumps",
- UI_KEY_GROWNCLUMPS,
- 0,
- 0,
- "Save grown clumps instead of original.",
- UI_GROUP_SEGMENTATION,
- &p->grownclumps,
- GAL_OPTIONS_NO_ARG_TYPE,
- GAL_OPTIONS_RANGE_0_OR_1,
- GAL_OPTIONS_NOT_MANDATORY,
- GAL_OPTIONS_NOT_SET
- },
- {
- "checksegmentation",
- UI_KEY_CHECKSEGMENTATION,
- 0,
- 0,
- "Store segmentation steps in a file.",
- UI_GROUP_SEGMENTATION,
- &p->checksegmentation,
- GAL_OPTIONS_NO_ARG_TYPE,
- GAL_OPTIONS_RANGE_0_OR_1,
- GAL_OPTIONS_NOT_MANDATORY,
- GAL_OPTIONS_NOT_SET
- },
-
-
-
-
-
/* Operating mode options. */
{
"continueaftercheck",
diff --git a/bin/noisechisel/astnoisechisel-3d.conf
b/bin/noisechisel/astnoisechisel-3d.conf
index fa4f4eb..46c3bbf 100644
--- a/bin/noisechisel/astnoisechisel-3d.conf
+++ b/bin/noisechisel/astnoisechisel-3d.conf
@@ -19,7 +19,7 @@
# Input:
khdu 1
- wkhdu 1
+ whdu 1
minskyfrac 0.7
minnumfalse 100
@@ -41,18 +41,10 @@
openingngb 18
sigmaclip 3,0.2
dthresh -0.1
- detsnminarea 10
- detquant 0.90
+ snminarea 10
+ snquant 0.90
detgrowquant 0.95
detgrowmaxholesize 300
-# Segmentation
- segsnminarea 15
- keepmaxnearriver 0
- segquant 0.95
- gthresh 0.5
- minriverlength 15
- objbordersn 1
-
# Operating mode
continueaftercheck 0
diff --git a/bin/noisechisel/astnoisechisel.conf
b/bin/noisechisel/astnoisechisel.conf
index 71bdd47..5126341 100644
--- a/bin/noisechisel/astnoisechisel.conf
+++ b/bin/noisechisel/astnoisechisel.conf
@@ -19,8 +19,8 @@
# Input:
khdu 1
- wkhdu 1
- convolvedhdu 1
+ whdu 1
+ chdu 1
minskyfrac 0.7
minnumfalse 100
@@ -40,18 +40,10 @@
openingngb 8
sigmaclip 3,0.2
dthresh 0.0
- detsnminarea 10
- detquant 0.95
- detgrowquant 0.70
+ snminarea 10
+ snquant 0.95
+ detgrowquant 0.80
detgrowmaxholesize 100
-# Segmentation
- segsnminarea 15
- keepmaxnearriver 0
- segquant 0.95
- gthresh 0.5
- minriverlength 15
- objbordersn 1
-
# Operating mode
continueaftercheck 0
diff --git a/bin/noisechisel/authors-cite.h b/bin/noisechisel/authors-cite.h
index 782d42d..81fe4b2 100644
--- a/bin/noisechisel/authors-cite.h
+++ b/bin/noisechisel/authors-cite.h
@@ -1,5 +1,5 @@
/*********************************************************************
-NoiseChisel - Detect and segment signal in a noisy dataset.
+NoiseChisel - Detect signal in a noisy dataset.
NoiseChisel is part of GNU Astronomy Utilities (Gnuastro) package.
Original author:
diff --git a/bin/noisechisel/detection.c b/bin/noisechisel/detection.c
index d924715..3894141 100644
--- a/bin/noisechisel/detection.c
+++ b/bin/noisechisel/detection.c
@@ -1,9 +1,9 @@
/*********************************************************************
-NoiseChisel - Detect and segment signal in a noisy dataset.
+NoiseChisel - Detect signal in a noisy dataset.
NoiseChisel is part of GNU Astronomy Utilities (Gnuastro) package.
Original author:
- Mohammad Akhlaghi <akhlaghi@gnu.org>
+ Mohammad Akhlaghi <mohammad@akhlaghi.org>
Contributing author(s):
Copyright (C) 2015-2018, Free Software Foundation, Inc.
@@ -29,18 +29,19 @@ along with Gnuastro. If not, see
<http://www.gnu.org/licenses/>.
#include <string.h>
#include <gnuastro/fits.h>
+#include <gnuastro/label.h>
#include <gnuastro/binary.h>
#include <gnuastro/threads.h>
#include <gnuastro/dimension.h>
#include <gnuastro/statistics.h>
#include <gnuastro-internal/timing.h>
+#include <gnuastro-internal/checkset.h>
#include "main.h"
#include "ui.h"
#include "sky.h"
-#include "clumps.h"
#include "threshold.h"
@@ -526,7 +527,7 @@ detection_sn(struct noisechiselparams *p, gal_data_t
*worklab, size_t num,
sn = gal_data_alloc(NULL, GAL_TYPE_FLOAT32, 1, &tablen, NULL, 1,
p->cp.minmapsize, "SIGNAL-TO-NOISE", "ratio",
NULL);
- snind = ( p->checkdetsn==0 ? NULL
+ snind = ( p->checksn==0 ? NULL
: gal_data_alloc(NULL, GAL_TYPE_INT32, 1, &tablen, NULL, 1,
p->cp.minmapsize, "LABEL", "counter",
NULL) );
@@ -601,7 +602,7 @@ detection_sn(struct noisechiselparams *p, gal_data_t
*worklab, size_t num,
plabend = (plab=worklab->array) + worklab->size;
do
if( *plab!=GAL_BLANK_INT32
- && ( area[*plab]<p->detsnminarea || brightness[*plab]<0) )
+ && ( area[*plab]<p->snminarea || brightness[*plab]<0) )
*plab=0;
while(++plab<plabend);
}
@@ -618,7 +619,7 @@ detection_sn(struct noisechiselparams *p, gal_data_t
*worklab, size_t num,
for(i=1;i<tablen;++i)
{
ave=brightness[i]/area[i];
- if( area[i]>p->detsnminarea && ave>0.0f && pos[i*pcols]>0.0f )
+ if( area[i]>p->snminarea && ave>0.0f && pos[i*pcols]>0.0f )
{
/* Get the flux weighted center coordinates. */
for(j=0;j<ndim;++j)
@@ -630,10 +631,6 @@ detection_sn(struct noisechiselparams *p, gal_data_t
*worklab, size_t num,
err = ((float *)(p->std->array))[
gal_tile_full_id_from_coord(&p->cp.tl, coord) ];
- /* If the image was already sky subtracted, the second power of
- the error needs to be doubled. */
- err *= p->skysubtracted ? err : 2.0f*err;
-
/* Correct the index in the sn to store the Signal to noise
ratio. When we are dealing with the noise, we only want the
non-zero signal to noise values, so we will just use a
@@ -643,7 +640,7 @@ detection_sn(struct noisechiselparams *p, gal_data_t
*worklab, size_t num,
ind = s0d1D2 ? i : counter++;
if(snind) indarr[ind]=i;
snarr[ind] = ( sqrt( (float)(area[i])/p->cpscorr )
- * ave / sqrt(ave+err) );
+ * ave / sqrt( ave + err*err ) );
}
else
/* In detection pseudo-detections, order matters, so we will set
@@ -761,19 +758,19 @@ detection_pseudo_real(struct noisechiselparams *p)
error(EXIT_FAILURE, 0, "only %zu pseudo-detections could be found over "
"the sky region to estimate an S/N. This is less than %zu (value "
"to `--minnumfalse' option). Please adjust parameters like "
- "`--dthresh', `--detsnminarea', or make sure that there actually "
+ "`--dthresh', `--snminarea', or make sure that there actually "
"is sufficient sky area after initial detection. You can use "
"`--checkdetection' to see every step until this point", sn->size,
p->minnumfalse);
/* Get the S/N quantile and report it if we are in non-quiet mode. */
- quant=gal_statistics_quantile(sn, p->detquant, 1);
+ quant=gal_statistics_quantile(sn, p->snquant, 1);
p->detsnthresh = *((float *)(quant->array));
if(!p->cp.quiet)
{
- if( asprintf(&msg, "Pseudo-det S/N: %.2f (%.2f quant of %zu).",
- p->detsnthresh, p->detquant, sn->size)<0 )
+ if( asprintf(&msg, "Pseudo-det S/N: %.3f (%g quant of %zu).",
+ p->detsnthresh, p->snquant, sn->size)<0 )
error(EXIT_FAILURE, 0, "%s: asprintf allocation", __func__);
gal_timing_report(&t1, msg, 2);
free(msg);
@@ -1034,7 +1031,7 @@ detection_quantile_expand(struct noisechiselparams *p,
gal_data_t *workbin)
/* If the binary value is 1, then we want an initial label of 1
(the object is already detected). If it isn't, then we only
want it if it is above the threshold. */
- *o = *b==1 ? 1 : ( *arr>*e_th ? CLUMPS_INIT : 0);
+ *o = *b==1 ? 1 : ( *arr>*e_th ? GAL_LABEL_INIT : 0);
if(*b==0 && *arr>*e_th)
*d++ = o - (int32_t *)(p->olabel->array);
@@ -1046,7 +1043,7 @@ detection_quantile_expand(struct noisechiselparams *p,
gal_data_t *workbin)
while(++o<of);
/* Expand the detections. */
- clumps_grow(p->olabel, diffuseindexs, 0, p->olabel->ndim);
+ gal_label_grow_indexs(p->olabel, diffuseindexs, 0, p->olabel->ndim);
/* Only keep the 1 valued pixels in the binary array and fill its
holes. */
@@ -1162,28 +1159,22 @@ detection(struct noisechiselparams *p)
/* Update the user on the progress, if necessary. */
- if(!p->cp.quiet)
+ if(!p->cp.quiet && p->detgrowquant!=1.0f && num_expanded!=GAL_BLANK_SIZE_T )
{
- if(p->detgrowquant==1.0f)
+ /* If the user hasn't asked for a labeled image, then don't confuse
+ them with the number of detections, just let them know that growth
+ is complete. */
+ if(p->label)
{
- if( asprintf(&msg, "%zu detections with no growth.",
- num_true_initial)<0 )
+ if( asprintf(&msg, "%zu detections after growth to %.3f "
+ "quantile.", num_true_initial, p->detgrowquant)<0 )
error(EXIT_FAILURE, 0, "%s: asprintf allocation", __func__);
}
else
{
- if(num_expanded==GAL_BLANK_SIZE_T)
- {
- if(asprintf(&msg, "%zu detections (no growth to %.3f "
- "quantile).", num_true_initial, p->detgrowquant)<0)
- error(EXIT_FAILURE, 0, "%s: asprintf allocation", __func__);
- }
- else
- {
- if( asprintf(&msg, "%zu detections after growth to %.3f "
- "quantile.", num_true_initial, p->detgrowquant)<0 )
- error(EXIT_FAILURE, 0, "%s: asprintf allocation", __func__);
- }
+ if( asprintf(&msg, "Growth to %.3f quantile complete.",
+ p->detgrowquant)<0 )
+ error(EXIT_FAILURE, 0, "%s: asprintf allocation", __func__);
}
gal_timing_report(&t1, msg, 2);
free(msg);
@@ -1202,8 +1193,17 @@ detection(struct noisechiselparams *p)
: num_true_initial );
if(!p->cp.quiet)
{
- if( asprintf(&msg, "%zu final true detections.", p->numdetections)<0 )
- error(EXIT_FAILURE, 0, "%s: asprintf allocation", __func__);
+ /* If the user hasn't asked for a labeled image, then don't confuse
+ them with the number of detections, just let them know that growth
+ is complete. */
+ if(p->label)
+ {
+ if( asprintf(&msg, "%zu final true detections.",
+ p->numdetections)<0 )
+ error(EXIT_FAILURE, 0, "%s: asprintf allocation", __func__);
+ }
+ else
+ gal_checkset_allocate_copy("Detection complete.", &msg);
gal_timing_report(&t0, msg, 1);
free(msg);
}
diff --git a/bin/noisechisel/detection.h b/bin/noisechisel/detection.h
index df679b1..fedfcf6 100644
--- a/bin/noisechisel/detection.h
+++ b/bin/noisechisel/detection.h
@@ -1,5 +1,5 @@
/*********************************************************************
-NoiseChisel - Detect and segment signal in a noisy dataset.
+NoiseChisel - Detect signal in a noisy dataset.
NoiseChisel is part of GNU Astronomy Utilities (Gnuastro) package.
Original author:
diff --git a/bin/noisechisel/kernel-3d.h b/bin/noisechisel/kernel-3d.h
deleted file mode 100644
index 96a274b..0000000
--- a/bin/noisechisel/kernel-3d.h
+++ /dev/null
@@ -1,50 +0,0 @@
-size_t kernel_3d_dsize[3]={5, 9, 9};
-float kernel_3d[405]={0, 0, 0, 0, 0, 0, 0, 0, 0,
-0, 0, 0, 0, 0, 0, 0, 0, 0,
-0, 0, 0, 0, 0, 0, 0, 0, 0,
-0, 0, 0, 0, 0, 0, 0, 0, 0,
-0, 0, 0, 0, 0, 0, 0, 0, 0,
-0, 0, 0, 0, 0, 0, 0, 0, 0,
-0, 0, 0, 0, 0, 0, 0, 0, 0,
-0, 0, 0, 0, 0, 0, 0, 0, 0,
-0, 0, 0, 0, 0, 0, 0, 0, 0,
-
-0, 0, 0, 0, 0, 0, 0, 0, 0,
-0, 0, 0, 5.293481e-07, 1.391654e-06, 5.283476e-07, 0, 0, 0,
-0, 0, 5.023404e-06, 0.0001093707, 0.0003028791, 0.0001103754, 5.206822e-06, 0,
0,
-0, 5.27038e-07, 0.0001140605, 0.002517939, 0.006957374, 0.002468025,
0.0001131454, 5.31773e-07, 0,
-0, 1.481738e-06, 0.0002994444, 0.006708808, 0.01894199, 0.006776441,
0.0003116253, 1.493134e-06, 0,
-0, 5.293459e-07, 0.0001081432, 0.002473192, 0.006704316, 0.002578867,
0.0001105525, 4.866729e-07, 0,
-0, 0, 4.985141e-06, 0.000110672, 0.0002885369, 0.0001184523, 4.688276e-06, 0,
0,
-0, 0, 0, 5.610833e-07, 1.336052e-06, 5.216995e-07, 0, 0, 0,
-0, 0, 0, 0, 0, 0, 0, 0, 0,
-
-0, 0, 0, 0, 0, 0, 0, 0, 0,
-0, 0, 3.462786e-07, 7.853481e-06, 2.153742e-05, 7.564613e-06, 3.611804e-07, 0,
0,
-0, 3.52808e-07, 7.161698e-05, 0.001676089, 0.004618002, 0.001656355,
7.414426e-05, 3.43867e-07, 0,
-0, 7.990315e-06, 0.001704389, 0.03857445, 0.1031222, 0.03779064, 0.001703231,
7.975499e-06, 0,
-0, 2.231775e-05, 0.00473238, 0.1043504, 0.2858114, 0.10241, 0.004618072,
2.193002e-05, 0,
-0, 7.621142e-06, 0.001647367, 0.03755038, 0.1036048, 0.03796006, 0.001672143,
7.758124e-06, 0,
-0, 3.68421e-07, 7.659229e-05, 0.001695716, 0.004599371, 0.001670764,
7.450273e-05, 3.481919e-07, 0,
-0, 0, 3.521386e-07, 7.924394e-06, 2.207155e-05, 7.889852e-06, 3.53245e-07, 0,
0,
-0, 0, 0, 0, 0, 0, 0, 0, 0,
-
-0, 0, 0, 0, 0, 0, 0, 0, 0,
-0, 0, 0, 5.182329e-07, 1.397928e-06, 5.17901e-07, 0, 0, 0,
-0, 0, 4.812169e-06, 0.0001135803, 0.0003043477, 0.000109481, 4.974718e-06, 0,
0,
-0, 5.377616e-07, 0.0001073098, 0.002424047, 0.006817353, 0.002492242,
0.0001147361, 5.33439e-07, 0,
-0, 1.44089e-06, 0.0002933802, 0.006909565, 0.01873765, 0.006651767,
0.0003042257, 1.366647e-06, 0,
-0, 5.166781e-07, 0.0001119049, 0.002454791, 0.006940499, 0.002578837,
0.0001061706, 5.329928e-07, 0,
-0, 0, 4.656712e-06, 0.0001124511, 0.0003010639, 0.0001131462, 4.786908e-06, 0,
0,
-0, 0, 0, 5.303103e-07, 1.363214e-06, 5.122773e-07, 0, 0, 0,
-0, 0, 0, 0, 0, 0, 0, 0, 0,
-
-0, 0, 0, 0, 0, 0, 0, 0, 0,
-0, 0, 0, 0, 0, 0, 0, 0, 0,
-0, 0, 0, 0, 0, 0, 0, 0, 0,
-0, 0, 0, 0, 0, 0, 0, 0, 0,
-0, 0, 0, 0, 0, 0, 0, 0, 0,
-0, 0, 0, 0, 0, 0, 0, 0, 0,
-0, 0, 0, 0, 0, 0, 0, 0, 0,
-0, 0, 0, 0, 0, 0, 0, 0, 0,
-0, 0, 0, 0, 0, 0, 0, 0, 0};
diff --git a/bin/noisechisel/main.c b/bin/noisechisel/main.c
index c49c91b..00479e9 100644
--- a/bin/noisechisel/main.c
+++ b/bin/noisechisel/main.c
@@ -1,5 +1,5 @@
/*********************************************************************
-NoiseChisel - Detect and segment signal in a noisy dataset.
+NoiseChisel - Detect signal in a noisy dataset.
NoiseChisel is part of GNU Astronomy Utilities (Gnuastro) package.
Original author:
diff --git a/bin/noisechisel/main.h b/bin/noisechisel/main.h
index ed78ef1..05eb729 100644
--- a/bin/noisechisel/main.h
+++ b/bin/noisechisel/main.h
@@ -1,5 +1,5 @@
/*********************************************************************
-NoiseChisel - Detect and segment signal in a noisy dataset.
+NoiseChisel - Detect signal in a noisy dataset.
NoiseChisel is part of GNU Astronomy Utilities (Gnuastro) package.
Original author:
@@ -45,24 +45,22 @@ struct noisechiselparams
struct gal_tile_two_layer_params ltl;/* Large tessellation. */
char *inputname; /* Input filename. */
char *kernelname; /* Input kernel filename. */
- char *widekernelname; /* Name of wider kernel to be used. */
- char *convolvedname; /* Convolved image (to avoid convolution).*/
- char *convolvedhdu; /* HDU of convolved image. */
char *khdu; /* Kernel HDU. */
- char *wkhdu; /* Wide kernel HDU. */
- uint8_t skysubtracted; /* Input has been Sky subtracted before. */
- float minskyfrac; /* Undetected area min. frac. in tile. */
- size_t minnumfalse; /* Min No. of det/seg for true quantile. */
+ char *convolvedname; /* Convolved image (to avoid convolution).*/
+ char *chdu; /* HDU of convolved image. */
+ char *widekernelname; /* Name of wider kernel to be used. */
+ char *whdu; /* Wide kernel HDU. */
- uint8_t onlydetection; /* Do not do any segmentation. */
- uint8_t grownclumps; /* Save grown clumps instead of original. */
uint8_t continueaftercheck; /* Don't abort after the check steps. */
+ uint8_t rawoutput; /* Only detection & 1 elem/tile output. */
+ uint8_t label; /* Label detections that are connected. */
float mirrordist; /* Maximum distance to check mode sym. */
float modmedqdiff; /* Difference between mode and median. */
+ float minskyfrac; /* Undetected area min. frac. in tile. */
float qthresh; /* Quantile threshold on convolved image. */
float qthreshtilequant; /* Remove tiles with lower quantile. */
- size_t smoothwidth; /* Width of flat kernel to smooth. */
+ size_t smoothwidth; /* Interpolation: flat kernel to smooth. */
uint8_t checkqthresh; /* Save the quantile threhsold steps. */
size_t erode; /* Number of erosions after thresholding. */
size_t erodengb; /* Connectivity for erosion. */
@@ -72,24 +70,16 @@ struct noisechiselparams
double sigmaclip[2]; /* Sigma-clipping parameters. */
uint8_t checkdetsky; /* Check pseudo-detection sky value. */
float dthresh; /* Sigma threshold for Pseudo-detections. */
- size_t detsnminarea; /* Minimum pseudo-detection area for S/N. */
- uint8_t checkdetsn; /* Save pseudo-detection S/N values. */
- float detquant; /* True detection quantile. */
+ size_t snminarea; /* Minimum pseudo-detection area for S/N. */
+ uint8_t checksn; /* Save pseudo-detection S/N values. */
+ size_t minnumfalse; /* Min No. of det/seg for true quantile. */
+ float snquant; /* True detection quantile. */
float detgrowquant; /* Quantile to grow true detections. */
size_t detgrowmaxholesize; /* Max. size of holes to fill in growth. */
uint8_t cleangrowndet; /* Remove grown objects with small S/N. */
uint8_t checkdetection; /* Save all detection steps to a file. */
uint8_t checksky; /* Check the Sky value estimation. */
- size_t segsnminarea; /* Minimum area for segmentation. */
- uint8_t checkclumpsn; /* Save the clump S/N values to a file. */
- float segquant; /* Quantile of clumps in sky for true S/N.*/
- uint8_t keepmaxnearriver; /* Keep clumps with a peak near a river. */
- float gthresh; /* Multiple of STD to stop growing clumps.*/
- size_t minriverlength; /* Min, len of good grown clump rivers. */
- float objbordersn; /* Minimum S/N for grown clumps to be one.*/
- uint8_t checksegmentation; /* Save the segmentation steps in file. */
-
/* Internal. */
char *qthreshname; /* Name of Quantile threshold check image.*/
char *detskyname; /* Name of Initial det sky check image. */
@@ -98,9 +88,6 @@ struct noisechiselparams
char *detsn_D_name; /* Final detection S/N name. */
char *detectionname; /* Name of detection steps file. */
char *skyname; /* Name of Sky estimation steps file. */
- char *clumpsn_s_name; /* Sky clump S/N name. */
- char *clumpsn_d_name; /* Detection clumps S/N name. */
- char *segmentationname; /* Name of segmentation steps file. */
gal_data_t *input; /* Input image. */
gal_data_t *kernel; /* Sharper kernel. */
@@ -109,7 +96,6 @@ struct noisechiselparams
gal_data_t *wconv; /* Convolved with wider kernel. */
gal_data_t *binary; /* For binary operations. */
gal_data_t *olabel; /* Labels of objects in the detection. */
- gal_data_t *clabel; /* Labels of clumps in the detection. */
gal_data_t *expand_thresh; /* Quantile threshold to expand per tile. */
gal_data_t *exp_thresh_full; /* Full array containing growth thresh. */
gal_data_t *sky; /* Mean of undetected pixels, per tile. */
@@ -128,10 +114,7 @@ struct noisechiselparams
size_t numinitialdets; /* Number of initial detections. */
size_t numdetections; /* Number of final detections. */
- size_t numclumps; /* Number of true clumps. */
- size_t numobjects; /* Number of objects. */
float detsnthresh; /* Pseudo-detection S/N threshold. */
- float clumpsnthresh; /* Clump S/N threshold. */
};
#endif
diff --git a/bin/noisechisel/noisechisel.c b/bin/noisechisel/noisechisel.c
index a9e4f60..04f6835 100644
--- a/bin/noisechisel/noisechisel.c
+++ b/bin/noisechisel/noisechisel.c
@@ -1,5 +1,5 @@
/*********************************************************************
-NoiseChisel - Detect and segment signal in a noisy dataset.
+NoiseChisel - Detect signal in a noisy dataset.
NoiseChisel is part of GNU Astronomy Utilities (Gnuastro) package.
Original author:
@@ -23,6 +23,8 @@ along with Gnuastro. If not, see
<http://www.gnu.org/licenses/>.
#include <config.h>
#include <stdio.h>
+#include <errno.h>
+#include <error.h>
#include <stdlib.h>
#include <gnuastro/fits.h>
@@ -38,7 +40,6 @@ along with Gnuastro. If not, see
<http://www.gnu.org/licenses/>.
#include "sky.h"
#include "detection.h"
#include "threshold.h"
-#include "segmentation.h"
@@ -117,52 +118,6 @@ noisechisel_convolve(struct noisechiselparams *p)
-static void
-noisechisel_find_sky_subtract(struct noisechiselparams *p)
-{
- /* Find the final Sky value. */
- sky_and_std(p, p->skyname);
-
- /* Abort if the user only wanted to see until this point.*/
- if(p->skyname && !p->continueaftercheck)
- ui_abort_after_check(p, p->skyname, NULL,
- "derivation of final Sky (and its STD) value");
-
- /* Subtract the Sky from the Input and Convolved (necessary for
- segmentation) images. */
- sky_subtract(p);
-}
-
-
-
-
-
-/* If convolution was not done while respecting channel edges (when there
- is more than one channel, pixels outside the edge weren't used in the
- convolution), then correct it. */
-static void
-noisechisel_convolve_correct_ch_edges(struct noisechiselparams *p)
-{
- struct gal_tile_two_layer_params *tl=&p->cp.tl;
-
- /* Correct the convolved image if necessary. */
- if( tl->totchannels>1 && tl->workoverch==0 )
- {
- /* Do the correction. */
- gal_convolve_spatial_correct_ch_edge(tl->tiles, p->kernel,
- p->cp.numthreads, 1, p->conv);
-
- /* Inform the user. */
- if(!p->cp.quiet)
- gal_timing_report(NULL, "Corrected convolution of touching channel "
- "edges", 1);
- }
-}
-
-
-
-
-
@@ -181,92 +136,47 @@ noisechisel_convolve_correct_ch_edges(struct
noisechiselparams *p)
/***********************************************************************/
/************* Output ***************/
/***********************************************************************/
-/* The input image has been sky subtracted for further processing. So we'll
- need to copy the input image directly into the output. */
-static void
-noisechisel_output_copy_input(struct noisechiselparams *p)
-{
- int status=0;
- fitsfile *in, *out;
- char card[FLEN_CARD];
-
-
- /* Create/open the output file. */
- out=gal_fits_open_to_write(p->cp.output);
-
-
- /* Open the input FITS file in the proper extension. */
- in=gal_fits_hdu_open(p->inputname, p->cp.hdu, READWRITE);
-
-
- /* Copy the input HDU into the output. */
- if( fits_copy_hdu(in, out, 0, &status) )
- gal_fits_io_error(status, "copying input hdu into first output hdu");
-
-
- /* If an extension name exists in the input HDU, then don't touch it. If
- the input doesn't have any, then make an EXTNAME keyword for it. Note
- that `fits_read_card' will return a non-zero if it doesn't find the
- keyword. */
- if( fits_read_card(out, "EXTNAME", card, &status) )
- {
- status=0;
- fits_write_key(out, TSTRING, "EXTNAME", "INPUT", "", &status);
- }
-
-
- /* Close the two files. */
- fits_close_file(in, &status);
- fits_close_file(out, &status);
-}
-
-
-
-
-
/* Write the output file. */
static void
noisechisel_output(struct noisechiselparams *p)
{
gal_fits_list_key_t *keys=NULL;
- /* Copy the input image into the first extension. */
- noisechisel_output_copy_input(p);
+
+ /* Put a copy of the input into the output (when necessary). */
+ if(p->rawoutput==0)
+ {
+ /* Subtract the Sky value. */
+ sky_subtract(p);
+
+ /* Correct the name of the input and write it out. */
+ if(p->input->name) free(p->input->name);
+ p->input->name="INPUT-NO-SKY";
+ gal_fits_img_write(p->input, p->cp.output, NULL, PROGRAM_NAME);
+ p->input->name=NULL;
+ }
/* Write the object labels and useful information into it's header. */
- if(p->onlydetection==0)
- gal_fits_key_list_add(&keys, GAL_TYPE_STRING, "WCLUMPS", 0, "yes", 0,
- "Generate catalog with clumps?", 0, "bool");
- gal_fits_key_list_add(&keys, GAL_TYPE_SIZE_T, "NUMLABS", 0,
- &p->numobjects, 0, "Total number of labels "
- "(inclusive)", 0, "counter");
gal_fits_key_list_add(&keys, GAL_TYPE_FLOAT32, "DETSN", 0, &p->detsnthresh,
0, "Minimum S/N of true pseudo-detections", 0,
"ratio");
- p->olabel->name = p->onlydetection ? "DETECTIONS" : "OBJECTS";
- gal_fits_img_write(p->olabel, p->cp.output, keys, PROGRAM_NAME);
- p->olabel->name=NULL;
- keys=NULL;
-
-
- /* Write the clumps labels and useful information into it's header. Note
- that to make the clumps image more easily viewable, we will set all
- sky pixels to blank. Only clump pixels that have an overlapping object
- pixel will be use anyway, so the sky pixels are irrelevant. */
- if(p->onlydetection==0)
+ if(p->label)
{
- p->clabel->name="CLUMPS";
gal_fits_key_list_add(&keys, GAL_TYPE_SIZE_T, "NUMLABS", 0,
- &p->numclumps, 0, "Total number of clumps", 0,
- "counter");
- gal_fits_key_list_add(&keys, GAL_TYPE_FLOAT32, "CLUMPSN", 0,
- &p->clumpsnthresh, 0, "Minimum S/N of true clumps",
- 0, "ratio");
- gal_fits_img_write(p->clabel, p->cp.output, keys, PROGRAM_NAME);
- p->clabel->name=NULL;
- keys=NULL;
+ &p->numdetections, 0, "Total number of labels "
+ "(inclusive)", 0, "counter");
+ p->olabel->name = "DETECTIONS";
+ gal_fits_img_write(p->olabel, p->cp.output, keys, PROGRAM_NAME);
+ p->olabel->name=NULL;
}
+ else
+ {
+ p->binary->name = "DETECTIONS";
+ gal_fits_img_write(p->binary, p->cp.output, keys, PROGRAM_NAME);
+ p->binary->name=NULL;
+ }
+ keys=NULL;
/* Write the Sky image into the output */
@@ -291,6 +201,10 @@ noisechisel_output(struct noisechiselparams *p)
gal_tile_full_values_write(p->std, &p->cp.tl, 1, p->cp.output, keys,
PROGRAM_NAME);
p->std->name=NULL;
+
+ /* Let the user know that the output is written. */
+ if(!p->cp.quiet)
+ printf(" - Output written to `%s'.\n", p->cp.output);
}
@@ -325,22 +239,28 @@ noisechisel(struct noisechiselparams *p)
/* Remove false detections. */
detection(p);
- /* Find the Sky value and subtract it from the input and convolved
- images. */
- noisechisel_find_sky_subtract(p);
-
- /* If the user only wanted detection, ignore the segmentation steps. */
- if(p->onlydetection==0)
+ /* If we have any detections, find the Sky value and subtract it from the
+ input and convolved images. */
+ if(p->numdetections)
{
- /* Correct the convolved image channel edges if necessary. */
- noisechisel_convolve_correct_ch_edges(p);
+ /* Find the final Sky and Sky STD values. */
+ sky_and_std(p, p->skyname);
- /* Do the segmentation. */
- segmentation(p);
+ /* Abort if the user only wanted to see until this point.*/
+ if(p->skyname && !p->continueaftercheck)
+ ui_abort_after_check(p, p->skyname, NULL,
+ "derivation of final Sky (and its STD) value");
+
+ /* Write the output. */
+ noisechisel_output(p);
}
else
- p->numobjects=p->numdetections;
-
- /* Write the output. */
- noisechisel_output(p);
+ {
+ if(p->cp.quiet)
+ error(0, 0, "no output file created: no detections could found "
+ "in `%s' with given parameters", p->inputname);
+ else
+ gal_timing_report(NULL, "NO OUTPUT FILE CREATED (try with "
+ "`--checkdetection' to see why)", 1);
+ }
}
diff --git a/bin/noisechisel/noisechisel.h b/bin/noisechisel/noisechisel.h
index 6fb6459..72376e5 100644
--- a/bin/noisechisel/noisechisel.h
+++ b/bin/noisechisel/noisechisel.h
@@ -1,5 +1,5 @@
/*********************************************************************
-NoiseChisel - Detect and segment signal in a noisy dataset.
+NoiseChisel - Detect signal in a noisy dataset.
NoiseChisel is part of GNU Astronomy Utilities (Gnuastro) package.
Original author:
diff --git a/bin/noisechisel/sky.c b/bin/noisechisel/sky.c
index b6e3ab5..ca0f35b 100644
--- a/bin/noisechisel/sky.c
+++ b/bin/noisechisel/sky.c
@@ -1,5 +1,5 @@
/*********************************************************************
-NoiseChisel - Detect and segment signal in a noisy dataset.
+NoiseChisel - Detect signal in a noisy dataset.
NoiseChisel is part of GNU Astronomy Utilities (Gnuastro) package.
Original author:
@@ -239,9 +239,8 @@ void
sky_subtract(struct noisechiselparams *p)
{
size_t tid;
- void *tarray=NULL;
+ gal_data_t *tile;
float *sky=p->sky->array;
- gal_data_t *tile, *tblock=NULL;
/* A small sanity check. */
if(p->sky->type!=GAL_TYPE_FLOAT32)
@@ -255,31 +254,7 @@ sky_subtract(struct noisechiselparams *p)
/* For easy reading. */
tile=&p->cp.tl.tiles[tid];
- /* First subtract the Sky value from the input image. */
+ /* Subtract the Sky value from the input image. */
GAL_TILE_PARSE_OPERATE(tile, NULL, 0, 0, {*i-=sky[tid];});
-
- /* Change to the convolved image (if there is any). */
- if(p->conv!=p->input)
- {
- tarray=tile->array;
- tblock=tile->block;
- tile->array=gal_tile_block_relative_to_other(tile, p->conv);
- tile->block=p->conv;
-
- /* The threshold is always low. So for the majority of non-NaN
- pixels in the image, the condition above will be true. If we
- come over a NaN pixel, then by definition of NaN, all
- conditionals will fail.
-
- If an image doesn't have any NaN pixels, only the pixels below
- the threshold have to be checked for a NaN which are by
- definition a very small fraction of the total pixels. And if
- there are NaN pixels in the image. */
- GAL_TILE_PARSE_OPERATE(tile, NULL, 0, 0, {*i-=sky[tid];});
-
- /* Revert back to the original block. */
- tile->array=tarray;
- tile->block=tblock;
- }
}
}
diff --git a/bin/noisechisel/sky.h b/bin/noisechisel/sky.h
index faf6502..eaf0b73 100644
--- a/bin/noisechisel/sky.h
+++ b/bin/noisechisel/sky.h
@@ -1,5 +1,5 @@
/*********************************************************************
-NoiseChisel - Detect and segment signal in a noisy dataset.
+NoiseChisel - Detect signal in a noisy dataset.
NoiseChisel is part of GNU Astronomy Utilities (Gnuastro) package.
Original author:
diff --git a/bin/noisechisel/threshold.c b/bin/noisechisel/threshold.c
index d55c9ae..592a745 100644
--- a/bin/noisechisel/threshold.c
+++ b/bin/noisechisel/threshold.c
@@ -1,5 +1,5 @@
/*********************************************************************
-NoiseChisel - Detect and segment signal in a noisy dataset.
+NoiseChisel - Detect signal in a noisy dataset.
NoiseChisel is part of GNU Astronomy Utilities (Gnuastro) package.
Original author:
@@ -196,10 +196,17 @@ threshold_write_sn_table(struct noisechiselparams *p,
gal_data_t *insn,
set of blank elements, but checking on the integer array is faster. */
if( gal_blank_present(inind, 1) )
{
+ /* Remove blank elements. */
ind=gal_data_copy(inind);
sn=gal_data_copy(insn);
gal_blank_remove(ind);
gal_blank_remove(sn);
+
+ /* A small sanity check. */
+ if(ind->size==0 || sn->size==0)
+ error(EXIT_FAILURE, 0, "%s: a bug! Please contact us at %s to fix "
+ "the problem. For some reason, all the elements in `ind' or "
+ "`sn' are blank", __func__, PACKAGE_BUGREPORT);
}
else
{
@@ -648,6 +655,8 @@ void
threshold_quantile_find_apply(struct noisechiselparams *p)
{
char *msg;
+ size_t nval;
+ gal_data_t *num;
struct timeval t1;
struct qthreshparams qprm;
struct gal_options_common_params *cp=&p->cp;
@@ -734,6 +743,38 @@ threshold_quantile_find_apply(struct noisechiselparams *p)
p->qthreshname);
+ /* Check if the number of acceptable tiles is more than the minimum
+ interpolated number. Since this is a common problem for users, it is
+ much more useful to do the check here rather than printing multiple
+ errors in parallel. */
+ num=gal_statistics_number(qprm.erode_th);
+ nval=((size_t *)(num->array))[0];
+ if( nval < cp->interpnumngb)
+ error(EXIT_FAILURE, 0, "%zu tiles can be used for interpolation of the "
+ "quantile threshold values over the full dataset. This is smaller "
+ "than the requested minimum value of %zu (value to the "
+ "`--interpnumngb' option).\n\n"
+ "There are several ways to address the problem. The best and most "
+ "highly recommended is to use a larger input if possible (when the "
+ "input is a crop from a larger dataset). If that is not the case, "
+ "or it doesn't solve the problem, you need to loosen the "
+ "parameters (and therefore cause scatter in the final result). "
+ "Thus don't loosen them too much. Recall that you can see all the "
+ "option values to Gnuastro's programs by appending `-P' to the "
+ "end of your command.\n\n"
+ " * Decrease `--interpnumngb' to be smaller than %zu.\n"
+ " * Slightly decrease `--tilesize' to have more tiles.\n"
+ " * Slightly increase `--modmedqdiff' to accept more tiles.\n\n"
+ "Try appending your command with `--checkqthresh' to see the "
+ "successful tiles (and get a feeling of the cause/solution. Note "
+ "that the output is a multi-extension FITS file).\n\n"
+ "To better understand this important step, please run the "
+ "following command (press `SPACE'/arrow-keys to navigate and "
+ "`Q' to return back to the command-line):\n\n"
+ " $ info gnuastro \"Quantifying signal in a tile\"\n",
+ nval, cp->interpnumngb, cp->interpnumngb);
+
+
/* Interpolate and smooth the derived values. */
threshold_interp_smooth(p, &qprm.erode_th, &qprm.noerode_th,
qprm.expand_th ? &qprm.expand_th : NULL,
@@ -747,7 +788,11 @@ threshold_quantile_find_apply(struct noisechiselparams *p)
/* Write the binary image if check is requested. */
if(p->qthreshname && !tl->oneelempertile)
- gal_fits_img_write(p->binary, p->qthreshname, NULL, PROGRAM_NAME);
+ {
+ p->binary->name="QTHRESH-APPLIED";
+ gal_fits_img_write(p->binary, p->qthreshname, NULL, PROGRAM_NAME);
+ p->binary->name=NULL;
+ }
/* Set the expansion quantile if necessary. */
diff --git a/bin/noisechisel/threshold.h b/bin/noisechisel/threshold.h
index 66b18c4..daa6dfc 100644
--- a/bin/noisechisel/threshold.h
+++ b/bin/noisechisel/threshold.h
@@ -1,5 +1,5 @@
/*********************************************************************
-NoiseChisel - Detect and segment signal in a noisy dataset.
+NoiseChisel - Detect signal in a noisy dataset.
NoiseChisel is part of GNU Astronomy Utilities (Gnuastro) package.
Original author:
diff --git a/bin/noisechisel/ui.c b/bin/noisechisel/ui.c
index 38e49d9..156dc7e 100644
--- a/bin/noisechisel/ui.c
+++ b/bin/noisechisel/ui.c
@@ -1,5 +1,5 @@
/*********************************************************************
-NoiseChisel - Detect and segment signal in a noisy dataset.
+NoiseChisel - Detect signal in a noisy dataset.
NoiseChisel is part of GNU Astronomy Utilities (Gnuastro) package.
Original author:
@@ -30,6 +30,7 @@ along with Gnuastro. If not, see
<http://www.gnu.org/licenses/>.
#include <gnuastro/wcs.h>
#include <gnuastro/fits.h>
+#include <gnuastro/array.h>
#include <gnuastro/blank.h>
#include <gnuastro/threads.h>
#include <gnuastro/dimension.h>
@@ -230,8 +231,8 @@ ui_read_check_only_options(struct noisechiselparams *p)
{
/* If the convolved option is given, then the convolved HDU is also
mandatory. */
- if(p->convolvedname && p->convolvedhdu==NULL)
- error(EXIT_FAILURE, 0, "no value given to `--convolvedhdu'. When the "
+ if(p->convolvedname && p->chdu==NULL)
+ error(EXIT_FAILURE, 0, "no value given to `--chdu'. When the "
"`--convolved' option is called (to specify a convolved image "
"and avoid convolution) it is mandatory to also specify a HDU "
"for it");
@@ -250,9 +251,9 @@ ui_read_check_only_options(struct noisechiselparams *p)
/* For the options that make tables, the table format option is
mandatory. */
- if( (p->checkdetsn || p->checkclumpsn) && p->cp.tableformat==0 )
+ if( p->checksn && p->cp.tableformat==0 )
error(EXIT_FAILURE, 0, "`--tableformat' is necessary with the "
- "`--checkdetsn' and `--checkclumpsn' options.\n"
+ "`--checksn' option.\n"
"Please see description for `--tableformat' after running the "
"following command for more information (use `SPACE' to go down "
"the page and `q' to return to the command-line):\n\n"
@@ -280,12 +281,12 @@ ui_read_check_only_options(struct noisechiselparams *p)
gal_checkset_check_file(p->widekernelname);
/* If its FITS, see if a HDU has been provided. */
- if( gal_fits_name_is_fits(p->widekernelname) && p->wkhdu==NULL )
- error(EXIT_FAILURE, 0, "no HDU specified for wide kernel. When the "
- "wide kernel is a FITS file, a HDU must also be specified. You "
- "can use the `--khdu' option and give it the HDU number "
- "(starting from zero), extension name, or anything "
- "acceptable by CFITSIO");
+ if( gal_fits_name_is_fits(p->widekernelname) && p->whdu==NULL )
+ error(EXIT_FAILURE, 0, "no HDU specified for the given wide kernel "
+ "(`%s'). When the wide kernel is a FITS file, a HDU must also "
+ "be specified. You can use the `--whdu' option and give it the "
+ "HDU number (starting from zero), extension name, or any "
+ "HDU identifier acceptable by CFITSIO", p->widekernelname);
}
}
@@ -355,7 +356,7 @@ ui_set_output_names(struct noisechiselparams *p)
}
else
p->cp.output=gal_checkset_automatic_output(&p->cp, p->inputname,
- "_labeled.fits");
+ "_detected.fits");
/* Tile check. */
if(p->cp.tl.checktiles)
@@ -373,7 +374,7 @@ ui_set_output_names(struct noisechiselparams *p)
"_detsky.fits");
/* Pseudo-detection S/N values. */
- if(p->checkdetsn)
+ if(p->checksn)
{
p->detsn_s_name=gal_checkset_automatic_output(&p->cp, basename,
( p->cp.tableformat==GAL_TABLE_FORMAT_TXT
@@ -389,27 +390,11 @@ ui_set_output_names(struct noisechiselparams *p)
/* Detection steps. */
if(p->checkdetection)
p->detectionname=gal_checkset_automatic_output(&p->cp, basename,
- "_det.fits");
+ "_detcheck.fits");
- /* Detection steps. */
+ /* Sky checks. */
if(p->checksky)
p->skyname=gal_checkset_automatic_output(&p->cp, basename, "_sky.fits");
-
- /* Clump S/N values. */
- if(p->checkclumpsn)
- {
- p->clumpsn_s_name=gal_checkset_automatic_output(&p->cp, basename,
- ( p->cp.tableformat==GAL_TABLE_FORMAT_TXT
- ? "_clumpsn_sky.txt" : "_clumpsn_sky.fits") );
- p->clumpsn_d_name=gal_checkset_automatic_output(&p->cp, basename,
- ( p->cp.tableformat==GAL_TABLE_FORMAT_TXT
- ? "_clumpsn_det.txt" : "_clumpsn_det.fits") );
- }
-
- /* Segmentation steps. */
- if(p->checksegmentation)
- p->segmentationname=gal_checkset_automatic_output(&p->cp, basename,
- "_seg.fits");
}
@@ -417,120 +402,19 @@ ui_set_output_names(struct noisechiselparams *p)
/* Prepare the kernel, either from a file, or from the default arrays
- (depending on the dimensionality). The default kernels were created as
- follows:
-
- 2D: The following commands were put in a script and run. It will create
- a plain text array along with a FITS image.
-
- set -o errexit # Stop if a program returns false.
- echo "0 0.0 0.0 3 2 0 0 1 1 5" > tmp.txt
- export GSL_RNG_TYPE=ranlxs2
- export GSL_RNG_SEED=1
- astmkprof tmp.txt --oversample=1 --envseed --numrandom=10000 \
- --tolerance=0.01 --nomerged
- astcrop 0_tmp.fits --section=2:*-1,2:*-1 --zeroisnotblank \
- --output=fwhm2.fits
- astconvertt fwhm2.fits --output=fwhm2.txt
- rm 0_tmp.fits tmp.txt
-
- 3D: MakeProfiles was initially run with these commands:
-
- $ export GSL_RNG_SEED=1
- $ export GSL_RNG_TYPE=ranlxs2
- $ astmkprof --kernel=gaussian-3d,1.5,5,0.5 --oversample=1 --envseed
-
- The resulting fits file was converted to text with the this C program to
- generate the `kernel-3d.h' header file (until ConvertType supports 3D
- datasets) which is then "include"d into this function.
-
- #include <stdio.h>
- #include <stdlib.h>
- #include <gnuastro/fits.h>
-
- int
- main(void)
- {
- size_t i;
- float *arr;
- gal_data_t *img=gal_fits_img_read_to_type("kernel.fits", "1",
- GAL_TYPE_FLOAT32, -1);
-
- arr=img->array;
-
- printf("size_t kernel_3d_dsize[3]={%zu, %zu, %zu};\n",
- img->dsize[0], img->dsize[1], img->dsize[2]);
- printf("float kernel_3d[%zu]={", img->size);
- for(i=0;i<img->size;++i)
- {
- if(i>0)
- {
- if(i % img->dsize[2] == 0 ) printf("\n");
- if(i % (img->dsize[2]*img->dsize[1]) == 0 ) printf("\n");
- }
-
- // We cannot use `\b' here, since we are writing directly
- // to the command-line, so we'll first write the number,
- // then decide if any subsequent character (a comma)
- // should be written.
- printf("%.7g", arr[i]);
-
- // The last element doesn't need a comma. In each line,
- // the last character must not be a space, but for easy
- // readability, the elements in between need a space.
- if( i!=(img->size-1) )
- printf("%s", ((i+1)%img->dsize[2]) ? ", " : ",");
- }
- printf("};\n");
-
- return EXIT_SUCCESS;
- }
-
- Assuming this C program is in a file named `kernel.c', it can be
- compiled, run and saved into `kernel-3d.h' with the command below:
-
- $ astbuildprog -q kernel.c > kernel-3d.h
-*/
+ available in the headers. The default kernels were created as
+ follows. */
static void
ui_prepare_kernel(struct noisechiselparams *p)
{
-#include <kernel-3d.h>
float *f, *ff, *k;
size_t ndim=p->input->ndim;
- size_t kernel_2d_dsize[2]={11,11};
- float kernel_2d[121]=
- {
- 0, 0, 0, 0, 0, 6.57699e-09, 0, 0, 0, 0, 0,
- 0, 0, 6.57699e-09, 2.10464e-07, 1.68371e-06, 3.36742e-06, 1.68371e-06,
- 2.10464e-07, 6.57699e-09, 0, 0,
-
- 0, 6.57699e-09, 8.41855e-07, 2.69394e-05, 0.000383569, 0.000717224,
- 0.000379782, 2.69394e-05, 8.41855e-07, 6.57699e-09, 0,
-
- 0, 2.10464e-07, 2.69394e-05, 0.00140714, 0.00888549, 0.016448,
- 0.00867408, 0.00138203, 2.69394e-05, 2.10464e-07, 0,
-
- 0, 1.68371e-06, 0.000381138, 0.00875434, 0.0573377, 0.106308, 0.0570693,
- 0.00891745, 0.000378914, 1.68371e-06, 0,
-
- 6.57699e-09, 3.36742e-06, 0.00071364, 0.0164971, 0.106865, 0.197316,
- 0.106787, 0.0166434, 0.000713827, 3.36742e-06, 6.57699e-09,
-
- 0, 1.68371e-06, 0.000215515, 0.00894112, 0.0573699, 0.106239, 0.0567907,
- 0.00901191, 0.000215515, 1.68371e-06, 0,
-
- 0, 2.10464e-07, 2.69394e-05, 0.00135085, 0.0089288, 0.0164171,
- 0.00879334, 0.0013622, 2.69394e-05, 2.10464e-07, 0,
-
- 0, 6.57699e-09, 8.41855e-07, 2.69394e-05, 0.000215515, 0.000724137,
- 0.000215515, 2.69394e-05, 8.41855e-07, 6.57699e-09, 0,
-
- 0, 0, 6.57699e-09, 2.10464e-07, 1.68371e-06, 3.36742e-06, 1.68371e-06,
- 2.10464e-07, 6.57699e-09, 0, 0,
-
- 0, 0, 0, 0, 0, 6.57699e-09, 0, 0, 0, 0, 0
- };
+/* Since the default kernel has to be identical between NoiseChisel and
+ Segment, we have defined it in a shared header file to be accessible by
+ both programs. */
+#include <gnuastro-internal/kernel-2d.h>
+#include <gnuastro-internal/kernel-3d.h>
/* If a kernel file is given, then use it. Otherwise, use the default
kernel. */
@@ -556,11 +440,10 @@ ui_prepare_kernel(struct noisechiselparams *p)
do *f=*k++; while(++f<ff);
}
-
/* If a wide kernel is given, then read it into memory. Otherwise, just
ignore it. */
if(p->widekernelname)
- p->widekernel=gal_fits_img_read_kernel(p->widekernelname, p->wkhdu,
+ p->widekernel=gal_fits_img_read_kernel(p->widekernelname, p->whdu,
p->cp.minmapsize);
}
@@ -655,9 +538,11 @@ ui_preparations_read_input(struct noisechiselparams *p)
char *option_name=NULL, *good_values=NULL;
/* Read the input as a single precision floating point dataset. */
- p->input = gal_fits_img_read_to_type(p->inputname, p->cp.hdu,
- GAL_TYPE_FLOAT32,
- p->cp.minmapsize, 0, 0);
+ p->input = gal_array_read_one_ch_to_type(p->inputname, p->cp.hdu,
+ GAL_TYPE_FLOAT32,
+ p->cp.minmapsize);
+ p->input->wcs = gal_wcs_read(p->inputname, p->cp.hdu, 0, 0,
+ &p->input->nwcs);
if(p->input->name==NULL)
gal_checkset_allocate_copy("INPUT", &p->input->name);
@@ -730,15 +615,15 @@ ui_preparations(struct noisechiselparams *p)
if(p->convolvedname)
{
/* Read the input convolved image. */
- p->conv = gal_fits_img_read_to_type(p->convolvedname, p->convolvedhdu,
- GAL_TYPE_FLOAT32, p->cp.minmapsize,
- 0, 0);
+ p->conv = gal_array_read_one_ch_to_type(p->convolvedname, p->chdu,
+ GAL_TYPE_FLOAT32,
+ p->cp.minmapsize);
/* Make sure the convolved image is the same size as the input. */
if( gal_data_dsize_is_different(p->input, p->conv) )
error(EXIT_FAILURE, 0, "%s (hdu %s), given to `--convolved' and "
"`--convolvehdu', is not the same size as NoiseChisel's "
- "input: %s (hdu: %s)", p->convolvedname, p->convolvedhdu,
+ "input: %s (hdu: %s)", p->convolvedname, p->chdu,
p->inputname, p->cp.hdu);
}
else
@@ -846,7 +731,7 @@ ui_read_check_inputs_setup(int argc, char *argv[],
printf(" - Input: %s (hdu: %s)\n", p->inputname, p->cp.hdu);
if(p->convolvedname)
printf(" - Convolved input: %s (hdu: %s)\n",
- p->convolvedname, p->convolvedhdu);
+ p->convolvedname, p->chdu);
else
{
if(p->kernelname)
@@ -867,7 +752,7 @@ ui_read_check_inputs_setup(int argc, char *argv[],
}
if(p->widekernelname)
printf(" - Wide Kernel: %s (hdu: %s)\n", p->widekernelname,
- p->wkhdu);
+ p->whdu);
}
}
@@ -948,9 +833,6 @@ ui_free_report(struct noisechiselparams *p, struct timeval
*t1)
if(p->detsn_s_name) free(p->detsn_s_name);
if(p->detsn_d_name) free(p->detsn_d_name);
if(p->detectionname) free(p->detectionname);
- if(p->clumpsn_s_name) free(p->clumpsn_s_name);
- if(p->clumpsn_d_name) free(p->clumpsn_d_name);
- if(p->segmentationname) free(p->segmentationname);
/* Free the allocated datasets. */
gal_data_free(p->sky);
@@ -960,7 +842,6 @@ ui_free_report(struct noisechiselparams *p, struct timeval
*t1)
gal_data_free(p->kernel);
gal_data_free(p->binary);
gal_data_free(p->olabel);
- gal_data_free(p->clabel);
gal_data_free(p->widekernel);
if(p->conv!=p->input) gal_data_free(p->conv);
diff --git a/bin/noisechisel/ui.h b/bin/noisechisel/ui.h
index 6e33e94..0c5e7b2 100644
--- a/bin/noisechisel/ui.h
+++ b/bin/noisechisel/ui.h
@@ -1,5 +1,5 @@
/*********************************************************************
-NoiseChisel - Detect and segment signal in a noisy dataset.
+NoiseChisel - Detect signal in a noisy dataset.
NoiseChisel is part of GNU Astronomy Utilities (Gnuastro) package.
Original author:
@@ -50,8 +50,8 @@ enum program_args_groups
/* Available letters for short options:
- a b f j l n u x z
- A H J W X Y
+ a b f g i j n u v x y z
+ A E G H J O W X Y
*/
enum option_keys_enum
{
@@ -59,7 +59,6 @@ enum option_keys_enum
UI_KEY_LARGETILESIZE = 'L',
UI_KEY_KERNEL = 'k',
UI_KEY_WIDEKERNEL = 'w',
- UI_KEY_SKYSUBTRACTED = 'E',
UI_KEY_MINSKYFRAC = 'B',
UI_KEY_MIRRORDIST = 'r',
UI_KEY_MODMEDQDIFF = 'Q',
@@ -68,27 +67,20 @@ enum option_keys_enum
UI_KEY_OPENING = 'p',
UI_KEY_SIGMACLIP = 's',
UI_KEY_DTHRESH = 'R',
- UI_KEY_DETSNMINAREA = 'i',
- UI_KEY_DETQUANT = 'c',
+ UI_KEY_SNMINAREA = 'm',
+ UI_KEY_SNQUANT = 'c',
UI_KEY_DETGROWQUANT = 'd',
- UI_KEY_SEGSNMINAREA = 'm',
- UI_KEY_SEGQUANT = 'g',
- UI_KEY_KEEPMAXNEARRIVER = 'v',
- UI_KEY_GTHRESH = 'G',
- UI_KEY_MINRIVERLENGTH = 'y',
- UI_KEY_OBJBORDERSN = 'O',
UI_KEY_CONTINUEAFTERCHECK = 'C',
+ UI_KEY_LABEL = 'l',
/* Only with long version (start with a value 1000, the rest will be set
automatically). */
UI_KEY_KHDU = 1000,
UI_KEY_CONVOLVED,
- UI_KEY_CONVOLVEDHDU,
- UI_KEY_WKHDU,
+ UI_KEY_CHDU,
+ UI_KEY_WHDU,
UI_KEY_MINNUMFALSE,
- UI_KEY_ONLYDETECTION,
- UI_KEY_GROWNCLUMPS,
UI_KEY_SMOOTHWIDTH,
UI_KEY_QTHRESHTILEQUANT,
UI_KEY_CHECKQTHRESH,
@@ -96,13 +88,12 @@ enum option_keys_enum
UI_KEY_NOERODEQUANT,
UI_KEY_OPENINGNGB,
UI_KEY_CHECKDETSKY,
- UI_KEY_CHECKDETSN,
+ UI_KEY_CHECKSN,
UI_KEY_DETGROWMAXHOLESIZE,
UI_KEY_CLEANGROWNDET,
UI_KEY_CHECKDETECTION,
UI_KEY_CHECKSKY,
- UI_KEY_CHECKCLUMPSN,
- UI_KEY_CHECKSEGMENTATION,
+ UI_KEY_RAWOUTPUT,
};
diff --git a/bin/TEMPLATE/Makefile.am b/bin/segment/Makefile.am
similarity index 85%
copy from bin/TEMPLATE/Makefile.am
copy to bin/segment/Makefile.am
index f784618..5bc28ca 100644
--- a/bin/TEMPLATE/Makefile.am
+++ b/bin/segment/Makefile.am
@@ -26,16 +26,16 @@ AM_CPPFLAGS = -I\$(top_srcdir)/bootstrapped/lib
## Program definition (name, linking, sources and headers)
-bin_PROGRAMS = astTEMPLATE
+bin_PROGRAMS = astsegment
-astTEMPLATE_LDADD = -lgnuastro
+astsegment_LDADD = -lgnuastro
-astTEMPLATE_SOURCES = main.c ui.c TEMPLATE.c
+astsegment_SOURCES = main.c ui.c segment.c clumps.c
-EXTRA_DIST = main.h authors-cite.h args.h ui.h TEMPLATE.h
+EXTRA_DIST = main.h authors-cite.h args.h ui.h segment.h clumps.h
## The configuration file (distribute and install).
## NOTE: the man page is created in doc/Makefile.am
-dist_sysconf_DATA = astTEMPLATE.conf
+dist_sysconf_DATA = astsegment.conf
diff --git a/bin/noisechisel/args.h b/bin/segment/args.h
similarity index 51%
copy from bin/noisechisel/args.h
copy to bin/segment/args.h
index 77cea84..280a0cc 100644
--- a/bin/noisechisel/args.h
+++ b/bin/segment/args.h
@@ -1,11 +1,11 @@
/*********************************************************************
-NoiseChisel - Detect and segment signal in a noisy dataset.
-NoiseChisel is part of GNU Astronomy Utilities (Gnuastro) package.
+Segment - Segment initial labels based on signal structure.
+Segment is part of GNU Astronomy Utilities (Gnuastro) package.
Original author:
Mohammad Akhlaghi <mohammad@akhlaghi.org>
Contributing author(s):
-Copyright (C) 2015-2018, Free Software Foundation, Inc.
+Copyright (C) 2018, Free Software Foundation, Inc.
Gnuastro is free software: you can redistribute it and/or modify it
under the terms of the GNU General Public License as published by the
@@ -28,507 +28,288 @@ along with Gnuastro. If not, see
<http://www.gnu.org/licenses/>.
-/* Options in argp_option format. */
+/* Array of acceptable options. */
struct argp_option program_options[] =
{
-
/* Input options. */
{
- "kernel",
- UI_KEY_KERNEL,
- "STR",
+ "sky",
+ UI_KEY_SKY,
+ "STR/FLT",
0,
- "Filename of kernel to convolve with input",
+ "Filename of Sky values image to subtract.",
GAL_OPTIONS_GROUP_INPUT,
- &p->kernelname,
+ &p->skyname,
GAL_TYPE_STRING,
GAL_OPTIONS_RANGE_ANY,
GAL_OPTIONS_NOT_MANDATORY,
GAL_OPTIONS_NOT_SET
},
{
- "khdu",
- UI_KEY_KHDU,
+ "skyhdu",
+ UI_KEY_SKYHDU,
"STR",
0,
- "HDU containing kernel image.",
+ "HDU containing Sky value to subtract.",
GAL_OPTIONS_GROUP_INPUT,
- &p->khdu,
+ &p->skyhdu,
GAL_TYPE_STRING,
GAL_OPTIONS_RANGE_ANY,
GAL_OPTIONS_NOT_MANDATORY,
GAL_OPTIONS_NOT_SET
},
{
- "convolved",
- UI_KEY_CONVOLVED,
- "STR",
+ "std",
+ UI_KEY_STD,
+ "STR/FLT",
0,
- "Convolved image file to avoid convolution.",
+ "Filename of Sky standard deviation.",
GAL_OPTIONS_GROUP_INPUT,
- &p->convolvedname,
+ &p->stdname,
GAL_TYPE_STRING,
GAL_OPTIONS_RANGE_ANY,
GAL_OPTIONS_NOT_MANDATORY,
GAL_OPTIONS_NOT_SET
},
{
- "convolvedhdu",
- UI_KEY_CONVOLVEDHDU,
+ "stdhdu",
+ UI_KEY_STDHDU,
"STR",
0,
- "HDU/extension of convolved image in file.",
+ "HDU containing Sky standard deviation.",
GAL_OPTIONS_GROUP_INPUT,
- &p->convolvedhdu,
+ &p->stdhdu,
GAL_TYPE_STRING,
GAL_OPTIONS_RANGE_ANY,
GAL_OPTIONS_NOT_MANDATORY,
GAL_OPTIONS_NOT_SET
},
{
- "widekernel",
- UI_KEY_WIDEKERNEL,
+ "variance",
+ UI_KEY_VARIANCE,
+ 0,
+ 0,
+ "STD input is actually variance.",
+ GAL_OPTIONS_GROUP_INPUT,
+ &p->variance,
+ GAL_OPTIONS_NO_ARG_TYPE,
+ GAL_OPTIONS_RANGE_0_OR_1,
+ GAL_OPTIONS_NOT_MANDATORY,
+ GAL_OPTIONS_NOT_SET
+ },
+ {
+ "detection",
+ UI_KEY_DETECTION,
"STR",
0,
- "Filename of wider kernel for better qthresh",
+ "Filename of detection image (to segment).",
GAL_OPTIONS_GROUP_INPUT,
- &p->widekernelname,
+ &p->detectionname,
GAL_TYPE_STRING,
GAL_OPTIONS_RANGE_ANY,
GAL_OPTIONS_NOT_MANDATORY,
GAL_OPTIONS_NOT_SET
},
{
- "wkhdu",
- UI_KEY_WKHDU,
+ "dhdu",
+ UI_KEY_DHDU,
"STR",
0,
- "HDU containing wide kernel image.",
+ "HDU containing detection image.",
GAL_OPTIONS_GROUP_INPUT,
- &p->wkhdu,
+ &p->dhdu,
GAL_TYPE_STRING,
GAL_OPTIONS_RANGE_ANY,
GAL_OPTIONS_NOT_MANDATORY,
GAL_OPTIONS_NOT_SET
},
{
- "skysubtracted",
- UI_KEY_SKYSUBTRACTED,
- 0,
+ "kernel",
+ UI_KEY_KERNEL,
+ "STR",
0,
- "Input is Sky subtracted (for error estimation).",
+ "Filename of kernel to convolve with input.",
GAL_OPTIONS_GROUP_INPUT,
- &p->skysubtracted,
- GAL_OPTIONS_NO_ARG_TYPE,
- GAL_OPTIONS_RANGE_0_OR_1,
+ &p->kernelname,
+ GAL_TYPE_STRING,
+ GAL_OPTIONS_RANGE_ANY,
GAL_OPTIONS_NOT_MANDATORY,
GAL_OPTIONS_NOT_SET
},
{
- "minskyfrac",
- UI_KEY_MINSKYFRAC,
- "FLT",
+ "khdu",
+ UI_KEY_KHDU,
+ "STR",
0,
- "Min. fraction of undetected area in tile.",
+ "HDU containing kernel image.",
GAL_OPTIONS_GROUP_INPUT,
- &p->minskyfrac,
- GAL_TYPE_FLOAT32,
- GAL_OPTIONS_RANGE_GE_0_LE_1,
- GAL_OPTIONS_MANDATORY,
+ &p->khdu,
+ GAL_TYPE_STRING,
+ GAL_OPTIONS_RANGE_ANY,
+ GAL_OPTIONS_NOT_MANDATORY,
GAL_OPTIONS_NOT_SET
},
{
- "minnumfalse",
- UI_KEY_MINNUMFALSE,
- "INT",
+ "convolved",
+ UI_KEY_CONVOLVED,
+ "STR",
0,
- "Minimum number for S/N estimation.",
+ "Convolved image file to avoid convolution.",
GAL_OPTIONS_GROUP_INPUT,
- &p->minnumfalse,
- GAL_TYPE_SIZE_T,
- GAL_OPTIONS_RANGE_GT_0,
- GAL_OPTIONS_MANDATORY,
+ &p->convolvedname,
+ GAL_TYPE_STRING,
+ GAL_OPTIONS_RANGE_ANY,
+ GAL_OPTIONS_NOT_MANDATORY,
GAL_OPTIONS_NOT_SET
},
-
-
- /* Tessellation. */
{
- "largetilesize",
- UI_KEY_LARGETILESIZE,
- "INT[,INT]",
+ "chdu",
+ UI_KEY_CHDU,
+ "STR",
0,
- "Sim. to --tilesize, but for larger tiles.",
- GAL_OPTIONS_GROUP_TESSELLATION,
- &p->ltl.tilesize,
- GAL_TYPE_SIZE_T,
- GAL_OPTIONS_RANGE_GT_0,
- GAL_OPTIONS_MANDATORY,
- GAL_OPTIONS_NOT_SET,
- gal_options_parse_sizes_reverse
+ "HDU/extension of convolved image in file.",
+ GAL_OPTIONS_GROUP_INPUT,
+ &p->chdu,
+ GAL_TYPE_STRING,
+ GAL_OPTIONS_RANGE_ANY,
+ GAL_OPTIONS_NOT_MANDATORY,
+ GAL_OPTIONS_NOT_SET
},
- /* Output options. */
+ /* Output. */
{
- "onlydetection",
- UI_KEY_ONLYDETECTION,
+ "rawoutput",
+ UI_KEY_RAWOUTPUT,
0,
0,
- "Stop at the end of detection.",
+ "Output only object and clump labels.",
GAL_OPTIONS_GROUP_OUTPUT,
- &p->onlydetection,
+ &p->rawoutput,
GAL_OPTIONS_NO_ARG_TYPE,
GAL_OPTIONS_RANGE_0_OR_1,
GAL_OPTIONS_NOT_MANDATORY,
GAL_OPTIONS_NOT_SET
},
-
-
-
-
-
- /* Detection. */
- {
- 0, 0, 0, 0,
- "Detection:",
- UI_GROUP_DETECTION
- },
- {
- "mirrordist",
- UI_KEY_MIRRORDIST,
- "FLT",
- 0,
- "Max. dist. (error multip.) to find mode.",
- UI_GROUP_DETECTION,
- &p->mirrordist,
- GAL_TYPE_FLOAT32,
- GAL_OPTIONS_RANGE_ANY,
- GAL_OPTIONS_MANDATORY,
- GAL_OPTIONS_NOT_SET
- },
- {
- "modmedqdiff",
- UI_KEY_MODMEDQDIFF,
- "FLT",
- 0,
- "Max. mode and median quant diff. per tile.",
- UI_GROUP_DETECTION,
- &p->modmedqdiff,
- GAL_TYPE_FLOAT32,
- GAL_OPTIONS_RANGE_GE_0,
- GAL_OPTIONS_MANDATORY,
- GAL_OPTIONS_NOT_SET
- },
- {
- "qthresh",
- UI_KEY_QTHRESH,
- "FLT",
- 0,
- "Quantile threshold on convolved image.",
- UI_GROUP_DETECTION,
- &p->qthresh,
- GAL_TYPE_FLOAT32,
- GAL_OPTIONS_RANGE_GE_0_LT_1,
- GAL_OPTIONS_MANDATORY,
- GAL_OPTIONS_NOT_SET
- },
- {
- "qthreshtilequant",
- UI_KEY_QTHRESHTILEQUANT,
- "FLT",
- 0,
- "Remove tiles at higher quantiles.",
- UI_GROUP_DETECTION,
- &p->qthreshtilequant,
- GAL_TYPE_FLOAT32,
- GAL_OPTIONS_RANGE_GE_0_LE_1,
- GAL_OPTIONS_MANDATORY,
- GAL_OPTIONS_NOT_SET
- },
- {
- "smoothwidth",
- UI_KEY_SMOOTHWIDTH,
- "INT",
- 0,
- "Flat kernel width to smooth interpolated.",
- UI_GROUP_DETECTION,
- &p->smoothwidth,
- GAL_TYPE_SIZE_T,
- GAL_OPTIONS_RANGE_0_OR_ODD,
- GAL_OPTIONS_MANDATORY,
- GAL_OPTIONS_NOT_SET
- },
{
- "checkqthresh",
- UI_KEY_CHECKQTHRESH,
+ "grownclumps",
+ UI_KEY_GROWNCLUMPS,
0,
0,
- "Save quantile threshold estimation in file.",
- UI_GROUP_DETECTION,
- &p->checkqthresh,
+ "Save grown clumps instead of original.",
+ UI_GROUP_SEGMENTATION,
+ &p->grownclumps,
GAL_OPTIONS_NO_ARG_TYPE,
GAL_OPTIONS_RANGE_0_OR_1,
GAL_OPTIONS_NOT_MANDATORY,
GAL_OPTIONS_NOT_SET
},
{
- "erode",
- UI_KEY_ERODE,
- "INT",
- 0,
- "Number of erosions after thresholding.",
- UI_GROUP_DETECTION,
- &p->erode,
- GAL_TYPE_SIZE_T,
- GAL_OPTIONS_RANGE_GE_0,
- GAL_OPTIONS_MANDATORY,
- GAL_OPTIONS_NOT_SET
- },
- {
- "erodengb",
- UI_KEY_ERODENGB,
- "INT",
- 0,
- "Connectivity in erosion (number of ngbs).",
- UI_GROUP_DETECTION,
- &p->erodengb,
- GAL_TYPE_SIZE_T,
- GAL_OPTIONS_RANGE_GT_0,
- GAL_OPTIONS_MANDATORY,
- GAL_OPTIONS_NOT_SET
- },
- {
- "noerodequant",
- UI_KEY_NOERODEQUANT,
- "FLT",
- 0,
- "Quantile for no erosion.",
- UI_GROUP_DETECTION,
- &p->noerodequant,
- GAL_TYPE_FLOAT32,
- GAL_OPTIONS_RANGE_GE_0_LE_1,
- GAL_OPTIONS_MANDATORY,
- GAL_OPTIONS_NOT_SET
- },
- {
- "opening",
- UI_KEY_OPENING,
- "INT",
- 0,
- "Depth of opening after erosion.",
- UI_GROUP_DETECTION,
- &p->opening,
- GAL_TYPE_SIZE_T,
- GAL_OPTIONS_RANGE_GT_0,
- GAL_OPTIONS_MANDATORY,
- GAL_OPTIONS_NOT_SET
- },
- {
- "openingngb",
- UI_KEY_OPENINGNGB,
- "INT",
- 0,
- "Connectivity in opening (number of ngbs).",
- UI_GROUP_DETECTION,
- &p->openingngb,
- GAL_TYPE_SIZE_T,
- GAL_OPTIONS_RANGE_GT_0,
- GAL_OPTIONS_MANDATORY,
- GAL_OPTIONS_NOT_SET
- },
- {
- "sigmaclip",
- UI_KEY_SIGMACLIP,
- "FLT,FLT",
- 0,
- "Sigma multiple and, tolerance or number.",
- UI_GROUP_DETECTION,
- &p->sigmaclip,
- GAL_TYPE_STRING,
- GAL_OPTIONS_RANGE_ANY,
- GAL_OPTIONS_MANDATORY,
- GAL_OPTIONS_NOT_SET,
- gal_options_read_sigma_clip
- },
- {
- "checkdetsky",
- UI_KEY_CHECKDETSKY,
+ "continueaftercheck",
+ UI_KEY_CONTINUEAFTERCHECK,
0,
0,
- "Save Sky value estimation for pseudo-dets.",
- UI_GROUP_DETECTION,
- &p->checkdetsky,
+ "Continue processing after checks.",
+ GAL_OPTIONS_GROUP_OPERATING_MODE,
+ &p->continueaftercheck,
GAL_OPTIONS_NO_ARG_TYPE,
GAL_OPTIONS_RANGE_0_OR_1,
GAL_OPTIONS_NOT_MANDATORY,
GAL_OPTIONS_NOT_SET
},
+
+
+
+
+
+ /* Tessellation. */
{
- "dthresh",
- UI_KEY_DTHRESH,
- "FLT",
- 0,
- "Sigma threshold for Pseudo-detections.",
- UI_GROUP_DETECTION,
- &p->dthresh,
- GAL_TYPE_FLOAT32,
- GAL_OPTIONS_RANGE_ANY,
- GAL_OPTIONS_MANDATORY,
- GAL_OPTIONS_NOT_SET
- },
- {
- "detsnminarea",
- UI_KEY_DETSNMINAREA,
- "INT",
+ "largetilesize",
+ UI_KEY_LARGETILESIZE,
+ "INT[,INT]",
0,
- "Min. pseudo-detection area for S/N dist.",
- UI_GROUP_DETECTION,
- &p->detsnminarea,
+ "Sim. to --tilesize, but for larger tiles.",
+ GAL_OPTIONS_GROUP_TESSELLATION,
+ &p->ltl.tilesize,
GAL_TYPE_SIZE_T,
- GAL_OPTIONS_RANGE_ANY,
+ GAL_OPTIONS_RANGE_GT_0,
GAL_OPTIONS_MANDATORY,
- GAL_OPTIONS_NOT_SET
- },
- {
- "checkdetsn",
- UI_KEY_CHECKDETSN,
- 0,
- 0,
- "Save pseudo-detection S/N values to a file.",
- UI_GROUP_DETECTION,
- &p->checkdetsn,
- GAL_OPTIONS_NO_ARG_TYPE,
- GAL_OPTIONS_RANGE_0_OR_1,
- GAL_OPTIONS_NOT_MANDATORY,
- GAL_OPTIONS_NOT_SET
+ GAL_OPTIONS_NOT_SET,
+ gal_options_parse_sizes_reverse
},
+
+
+
+
+ /* Segmentation options. */
{
- "detquant",
- UI_KEY_DETQUANT,
- "FLT",
- 0,
- "Quantile in pseudo-det. to define true.",
- UI_GROUP_DETECTION,
- &p->detquant,
- GAL_TYPE_FLOAT32,
- GAL_OPTIONS_RANGE_GT_0_LT_1,
- GAL_OPTIONS_MANDATORY,
- GAL_OPTIONS_NOT_SET
+ 0, 0, 0, 0,
+ "Segmentation:",
+ UI_GROUP_SEGMENTATION
},
{
- "detgrowquant",
- UI_KEY_DETGROWQUANT,
+ "minskyfrac",
+ UI_KEY_MINSKYFRAC,
"FLT",
0,
- "Minimum quant. to expand true detections.",
- UI_GROUP_DETECTION,
- &p->detgrowquant,
+ "Min. fraction of undetected area in tile.",
+ UI_GROUP_SEGMENTATION,
+ &p->minskyfrac,
GAL_TYPE_FLOAT32,
GAL_OPTIONS_RANGE_GE_0_LE_1,
GAL_OPTIONS_MANDATORY,
GAL_OPTIONS_NOT_SET
},
{
- "detgrowmaxholesize",
- UI_KEY_DETGROWMAXHOLESIZE,
+ "snminarea",
+ UI_KEY_SNMINAREA,
"INT",
0,
- "Max. area of holes after growth to fill.",
- UI_GROUP_DETECTION,
- &p->detgrowmaxholesize,
+ "Minimum area of clumps for S/N estimation.",
+ UI_GROUP_SEGMENTATION,
+ &p->snminarea,
GAL_TYPE_SIZE_T,
- GAL_OPTIONS_RANGE_GE_0,
+ GAL_OPTIONS_RANGE_GT_0,
GAL_OPTIONS_MANDATORY,
GAL_OPTIONS_NOT_SET
},
{
- "cleangrowndet",
- UI_KEY_CLEANGROWNDET,
- 0,
- 0,
- "Remove small S/N grown detections.",
- UI_GROUP_DETECTION,
- &p->cleangrowndet,
- GAL_OPTIONS_NO_ARG_TYPE,
- GAL_OPTIONS_RANGE_ANY,
- GAL_OPTIONS_NOT_MANDATORY,
- GAL_OPTIONS_NOT_SET
- },
- {
- "checkdetection",
- UI_KEY_CHECKDETECTION,
- 0,
- 0,
- "Save all the detection steps to a file.",
- UI_GROUP_DETECTION,
- &p->checkdetection,
- GAL_OPTIONS_NO_ARG_TYPE,
- GAL_OPTIONS_RANGE_0_OR_1,
- GAL_OPTIONS_NOT_MANDATORY,
- GAL_OPTIONS_NOT_SET
- },
- {
- "checksky",
- UI_KEY_CHECKSKY,
+ "checksn",
+ UI_KEY_CHECKSN,
0,
0,
- "Final sky and its STD steps in a file.",
- UI_GROUP_DETECTION,
- &p->checksky,
+ "Save clump S/N values into a file.",
+ UI_GROUP_SEGMENTATION,
+ &p->checksn,
GAL_OPTIONS_NO_ARG_TYPE,
GAL_OPTIONS_RANGE_0_OR_1,
GAL_OPTIONS_NOT_MANDATORY,
GAL_OPTIONS_NOT_SET
},
-
-
-
-
-
- /* Segmentation */
- {
- 0, 0, 0, 0,
- "Segmentation:",
- UI_GROUP_SEGMENTATION
- },
{
- "segsnminarea",
- UI_KEY_SEGSNMINAREA,
+ "minnumfalse",
+ UI_KEY_MINNUMFALSE,
"INT",
0,
- "Minimum area of clumps for S/N estimation.",
+ "Minimum number for S/N estimation.",
UI_GROUP_SEGMENTATION,
- &p->segsnminarea,
+ &p->minnumfalse,
GAL_TYPE_SIZE_T,
GAL_OPTIONS_RANGE_GT_0,
GAL_OPTIONS_MANDATORY,
GAL_OPTIONS_NOT_SET
},
{
- "checkclumpsn",
- UI_KEY_CHECKCLUMPSN,
- 0,
- 0,
- "Save Sky clump S/N values into a file.",
- UI_GROUP_SEGMENTATION,
- &p->checkclumpsn,
- GAL_OPTIONS_NO_ARG_TYPE,
- GAL_OPTIONS_RANGE_0_OR_1,
- GAL_OPTIONS_NOT_MANDATORY,
- GAL_OPTIONS_NOT_SET
- },
- {
- "segquant",
- UI_KEY_SEGQUANT,
+ "snquant",
+ UI_KEY_SNQUANT,
"FLT",
0,
"S/N Quantile of true sky clumps.",
UI_GROUP_SEGMENTATION,
- &p->segquant,
+ &p->snquant,
GAL_TYPE_FLOAT32,
GAL_OPTIONS_RANGE_GT_0,
GAL_OPTIONS_MANDATORY,
@@ -548,6 +329,19 @@ struct argp_option program_options[] =
GAL_OPTIONS_NOT_SET
},
{
+ "clumpsnthresh",
+ UI_KEY_CLUMPSNTHRESH,
+ "FLT",
+ 0,
+ "S/N threshold of true clumps.",
+ UI_GROUP_SEGMENTATION,
+ &p->clumpsnthresh,
+ GAL_TYPE_FLOAT32,
+ GAL_OPTIONS_RANGE_GT_0,
+ GAL_OPTIONS_NOT_MANDATORY,
+ GAL_OPTIONS_NOT_SET
+ },
+ {
"gthresh",
UI_KEY_GTHRESH,
"FLT",
@@ -587,19 +381,6 @@ struct argp_option program_options[] =
GAL_OPTIONS_NOT_SET
},
{
- "grownclumps",
- UI_KEY_GROWNCLUMPS,
- 0,
- 0,
- "Save grown clumps instead of original.",
- UI_GROUP_SEGMENTATION,
- &p->grownclumps,
- GAL_OPTIONS_NO_ARG_TYPE,
- GAL_OPTIONS_RANGE_0_OR_1,
- GAL_OPTIONS_NOT_MANDATORY,
- GAL_OPTIONS_NOT_SET
- },
- {
"checksegmentation",
UI_KEY_CHECKSEGMENTATION,
0,
@@ -618,25 +399,11 @@ struct argp_option program_options[] =
/* Operating mode options. */
- {
- "continueaftercheck",
- UI_KEY_CONTINUEAFTERCHECK,
- 0,
- 0,
- "Continue processing after checks.",
- GAL_OPTIONS_GROUP_OPERATING_MODE,
- &p->continueaftercheck,
- GAL_OPTIONS_NO_ARG_TYPE,
- GAL_OPTIONS_RANGE_0_OR_1,
- GAL_OPTIONS_NOT_MANDATORY,
- GAL_OPTIONS_NOT_SET
- },
- /* End of options. */
{0}
};
@@ -644,7 +411,10 @@ struct argp_option program_options[] =
-/* Define the child argp structure. */
+/* Define the child argp structure
+ -------------------------------
+
+ NOTE: these parts can be left untouched.*/
struct argp
gal_options_common_child = {gal_commonopts_options,
gal_options_common_argp_parse,
diff --git a/bin/segment/astsegment-3d.conf b/bin/segment/astsegment-3d.conf
new file mode 100644
index 0000000..2dcaed3
--- /dev/null
+++ b/bin/segment/astsegment-3d.conf
@@ -0,0 +1,43 @@
+# Default parameters (System) for Segment (in 3D).
+# Segment is part of GNU Astronomy Utilities.
+#
+# Use the long option name of each parameter followed by a value. The name
+# and value should be separated by atleast one white-space character (for
+# example ` '[space], or tab). Lines starting with `#' are ignored.
+#
+# For more information, please run these commands:
+#
+# $ astsegment --help # Full list of options, short doc.
+# $ astsegment -P # Print all options and used values.
+# $ info astsegment # All options and input/output.
+# $ info gnuastro "Configuration files" # How to use configuration files.
+#
+# Copying and distribution of this file, with or without modification, are
+# permitted in any medium without royalty provided the copyright notice and
+# this notice are preserved. This file is offered as-is, without any
+# warranty.
+
+# Input:
+ khdu 1
+ chdu 1
+ dhdu DETECTIONS
+ skyhdu SKY
+ stdhdu SKY_STD
+ minskyfrac 0.7
+ minnumfalse 100
+
+# Tessellation
+ numchannels 1,1,1
+ tilesize 15,15,15
+ largetilesize 50,50,50
+
+# Segmentation
+ snminarea 15
+ keepmaxnearriver 0
+ snquant 0.95
+ gthresh 0.5
+ minriverlength 15
+ objbordersn 1
+
+# Operating mode
+ continueaftercheck 0
diff --git a/bin/segment/astsegment.conf b/bin/segment/astsegment.conf
new file mode 100644
index 0000000..e5a3b7f
--- /dev/null
+++ b/bin/segment/astsegment.conf
@@ -0,0 +1,41 @@
+# Default parameters (System) for Segment.
+# Segment is part of GNU Astronomy Utilities.
+#
+# Use the long option name of each parameter followed by a value. The name
+# and value should be separated by atleast one white-space character (for
+# example ` '[space], or tab). Lines starting with `#' are ignored.
+#
+# For more information, please run these commands:
+#
+# $ astsegment --help # Full list of options, short doc.
+# $ astsegment -P # Print all options and used values.
+# $ info astsegment # All options and input/output.
+# $ info gnuastro "Configuration files" # How to use configuration files.
+#
+# Copying and distribution of this file, with or without modification, are
+# permitted in any medium without royalty provided the copyright notice and
+# this notice are preserved. This file is offered as-is, without any
+# warranty.
+
+# Input:
+ khdu 1
+ chdu 1
+ dhdu DETECTIONS
+ skyhdu SKY
+ stdhdu SKY_STD
+ minskyfrac 0.7
+ minnumfalse 100
+
+# Tessellation
+ largetilesize 200,200
+
+# Segmentation
+ snminarea 15
+ keepmaxnearriver 0
+ snquant 0.95
+ gthresh 0.5
+ minriverlength 15
+ objbordersn 1
+
+# Operating mode
+ continueaftercheck 0
diff --git a/bin/noisechisel/authors-cite.h b/bin/segment/authors-cite.h
similarity index 88%
copy from bin/noisechisel/authors-cite.h
copy to bin/segment/authors-cite.h
index 782d42d..8b6855b 100644
--- a/bin/noisechisel/authors-cite.h
+++ b/bin/segment/authors-cite.h
@@ -1,11 +1,11 @@
/*********************************************************************
-NoiseChisel - Detect and segment signal in a noisy dataset.
-NoiseChisel is part of GNU Astronomy Utilities (Gnuastro) package.
+Segment - Segment initial labels based on signal structure.
+Segment is part of GNU Astronomy Utilities (Gnuastro) package.
Original author:
Mohammad Akhlaghi <mohammad@akhlaghi.org>
Contributing author(s):
-Copyright (C) 2017-2018, Free Software Foundation, Inc.
+Copyright (C) 2018, Free Software Foundation, Inc.
Gnuastro is free software: you can redistribute it and/or modify it
under the terms of the GNU General Public License as published by the
diff --git a/bin/noisechisel/clumps.c b/bin/segment/clumps.c
similarity index 59%
rename from bin/noisechisel/clumps.c
rename to bin/segment/clumps.c
index cd2ce59..bd6b232 100644
--- a/bin/noisechisel/clumps.c
+++ b/bin/segment/clumps.c
@@ -1,6 +1,6 @@
/*********************************************************************
-NoiseChisel - Detect and segment signal in a noisy dataset.
-NoiseChisel is part of GNU Astronomy Utilities (Gnuastro) package.
+Segment - Segment initial labels based on signal structure.
+Segment is part of GNU Astronomy Utilities (Gnuastro) package.
Original author:
Mohammad Akhlaghi <mohammad@akhlaghi.org>
@@ -31,6 +31,7 @@ along with Gnuastro. If not, see
<http://www.gnu.org/licenses/>.
#include <gnuastro/fits.h>
#include <gnuastro/qsort.h>
#include <gnuastro/blank.h>
+#include <gnuastro/label.h>
#include <gnuastro/threads.h>
#include <gnuastro/dimension.h>
#include <gnuastro/statistics.h>
@@ -42,355 +43,6 @@ along with Gnuastro. If not, see
<http://www.gnu.org/licenses/>.
#include "ui.h"
#include "clumps.h"
-#include "threshold.h"
-
-
-
-
-
-
-
-
-
-
-/****************************************************************
- ***************** Over segmentation ********************
- ****************************************************************/
-/* Over-segment the region specified by its indexs into peaks and their
- respective regions (clumps). This is very similar to the immersion
- method of Vincent & Soille(1991), but here, we will not separate the
- image into layers, instead, we will work based on the ordered flux
- values. If a certain pixel (at a certain level) has no neighbors, it is
- a local maximum and will be assigned a new label. If it has a labeled
- neighbor, it will take that label and if there is more than one
- neighboring labeled region that pixel will be a `river` pixel. */
-void
-clumps_oversegment(struct clumps_thread_params *cltprm)
-{
- struct noisechiselparams *p=cltprm->clprm->p;
- size_t ndim=p->input->ndim;
-
- float *arr=p->conv->array;
- gal_data_t *indexs=cltprm->indexs;
- int connectivity=ndim==2?ndim:ndim-1;
- gal_list_sizet_t *Q=NULL, *cleanup=NULL;
- size_t *a, *af, ind, *dsize=p->input->dsize;
- size_t *dinc=gal_dimension_increment(ndim, dsize);
- int32_t n1, nlab, rlab, curlab=1, *clabel=p->clabel->array;
-
- /*********************************************
- For checks and debugging:*
- gal_data_t *crop;
- size_t extcount=1;
- int32_t *cr, *crf;
- size_t checkdsize[2]={10,10};
- size_t checkstart[2]={50,145};
- char *filename="clumpbuild.fits";
- size_t checkstartind=gal_dimension_coord_to_index(2, dsize, checkstart);
- gal_data_t *tile=gal_data_alloc(gal_data_ptr_increment(arr, checkstartind,
- p->conv->type),
- GAL_TYPE_INVALID, 2, checkdsize,
- NULL, 0, 0, NULL, NULL, NULL);
- tile->block=p->conv;
- gal_checkset_writable_remove(filename, 0, 0);
- if(p->cp.numthreads!=1)
- error(EXIT_FAILURE, 0, "in the debugging mode of `clumps_oversegment' "
- "only one thread must be used");
- crop=gal_data_copy(tile);
- gal_fits_img_write(crop, filename, NULL, PROGRAM_NAME);
- gal_data_free(crop);
- printf("blank: %u\nriver: %u\ntmpcheck: %u\ninit: %u\nmaxlab: %u\n",
- (int32_t)GAL_BLANK_INT32, (int32_t)CLUMPS_RIVER,
- (int32_t)CLUMPS_TMPCHECK, (int32_t)CLUMPS_INIT,
- (int32_t)CLUMPS_MAXLAB);
- tile->array=gal_tile_block_relative_to_other(tile, p->clabel);
- tile->block=p->clabel;
- **********************************************/
-
-
- /* If the size of the indexs is zero, then this function is pointless. */
- if(indexs->size==0) { cltprm->numinitclumps=0; return; }
-
-
- /* Sort the given indexs based on their flux (`gal_qsort_index_arr' is
- defined as static in `gnuastro/qsort.h') */
- gal_qsort_index_arr=p->conv->array;
- qsort(indexs->array, indexs->size, sizeof(size_t),
- gal_qsort_index_float_decreasing);
-
-
- /* Initialize the region we want to over-segment. */
- af=(a=indexs->array)+indexs->size; do clabel[*a]=CLUMPS_INIT; while(++a<af);
-
-
- /* Go over all the given indexs and pull out the clumps. */
- af=(a=indexs->array)+indexs->size;
- do
- /* When regions of a constant flux or masked regions exist, some later
- indexs (although they have same flux) will be filled before hand. If
- they are done, there is no need to do them again. */
- if(clabel[*a]==CLUMPS_INIT)
- {
- /* It might happen where one or multiple regions of the pixels
- under study have the same flux. So two equal valued pixels of
- two separate (but equal flux) regions will fall immediately
- after each other in the sorted list of indexs and we have to
- account for this.
-
- Therefore, if we see that the next pixel in the index list has
- the same flux as this one, it does not guarantee that it should
- be given the same label. Similar to the breadth first search
- algorithm for finding connected components, we will search all
- the neighbours and the neighbours of those neighbours that have
- the same flux of this pixel to see if they touch any label or
- not and to finally give them all the same label. */
- if( (a+1)<af && arr[*a]==arr[*(a+1)] )
- {
- /* Label of first neighbor found. */
- n1=0;
-
- /* A small sanity check. */
- if(Q!=NULL || cleanup!=NULL)
- error(EXIT_FAILURE, 0, "%s: a bug! Please contact us at %s so "
- "we can fix this problem. `Q' and `cleanup' should be "
- "NULL but while checking the equal flux regions they "
- "aren't", __func__, PACKAGE_BUGREPORT);
-
- /* Add this pixel to a queue. */
- gal_list_sizet_add(&Q, *a);
- gal_list_sizet_add(&cleanup, *a);
- clabel[*a] = CLUMPS_TMPCHECK;
-
- /* Find all the pixels that have the same flux and are
- connected. */
- while(Q!=NULL)
- {
- /* Pop an element from the queue. */
- ind=gal_list_sizet_pop(&Q);
-
- /* Look at the neighbors and see if we already have a
- label. */
- GAL_DIMENSION_NEIGHBOR_OP(ind, ndim, dsize, connectivity,
- dinc,
- {
- /* If it is already decided to be a river, then stop
- looking at the neighbors. */
- if(n1!=CLUMPS_RIVER)
- {
- /* For easy reading. */
- nlab=clabel[ nind ];
-
- /* This neighbor's label isn't zero. */
- if(nlab)
- {
- /* If this neighbor has not been labeled yet
- and has an equal flux, add it to the queue
- to expand the studied region.*/
- if( nlab==CLUMPS_INIT && arr[nind]==arr[*a] )
- {
- clabel[nind]=CLUMPS_TMPCHECK;
- gal_list_sizet_add(&Q, nind);
- gal_list_sizet_add(&cleanup, nind);
- }
- else
- n1=( nlab>0
-
- /* If this neighbor has a positive
- nlab, it belongs to another object,
- so if `n1' has not been set for the
- whole region (n1==0), put `nlab'
- into `n1'. If `n1' has been set and
- is different from `nlab' then this
- whole equal flux region should be a
- wide river because it is connecting
- two connected regions.*/
- ? ( n1
- ? (n1==nlab ? n1 : CLUMPS_RIVER)
- : nlab )
-
- /* If the data has blank pixels (recall
- that blank in int32 is negative),
- see if the neighbor is blank and if
- so, set the label to a river. Since
- the flag checking can be done
- outside this loop, for datasets with
- no blank element this last step will
- be completley ignored. */
- : ( ( (p->input->flag
- & GAL_DATA_FLAG_HASBLANK)
- && nlab==GAL_BLANK_INT32 )
- ? CLUMPS_RIVER : n1 ) );
- }
-
- /* If this neigbour has a label of zero, then we
- are on the edge of the indexed region (the
- neighbor is not in the initial list of pixels
- to segment). When over-segmenting the noise and
- the detections, `clabel' is zero for the parts
- of the image that we are not interested in
- here. */
- else clabel[*a]=CLUMPS_RIVER;
- }
- } );
- }
-
- /* Set the label that is to be given to this equal flux
- region. If `n1' was set to any value, then that label should
- be used for the whole region. Otherwise, this is a new
- label, see the case for a non-flat region. */
- if(n1) rlab = n1;
- else
- {
- rlab = curlab++;
- if( cltprm->topinds ) /* This is a local maximum of */
- cltprm->topinds[rlab]=*a; /* this region, save its index.*/
- }
-
- /* Give the same label to the whole connected equal flux
- region, except those that might have been on the side of
- the image and were a river pixel. */
- while(cleanup!=NULL)
- {
- ind=gal_list_sizet_pop(&cleanup);
- /* If it was on the sides of the image, it has been
- changed to a river pixel. */
- if( clabel[ ind ]==CLUMPS_TMPCHECK ) clabel[ ind ]=rlab;
- }
- }
-
- /* The flux of this pixel is not the same as the next sorted
- flux, so simply find the label for this object. */
- else
- {
- /* `n1' is the label of the first labeled neighbor found, so
- we'll initialize it to zero. */
- n1=0;
-
- /* Go over all the fully connected neighbors of this pixel and
- see if all the neighbors (with maximum connectivity: the
- number of dimensions) that have a non-macro value (less than
- CLUMPS_MAXLAB) belong to one label or not. If the pixel is
- neighboured by more than one label, set it as a river
- pixel. Also if it is touching a zero valued pixel (which
- does not belong to this object), set it as a river pixel.*/
- GAL_DIMENSION_NEIGHBOR_OP(*a, ndim, dsize, connectivity, dinc,
- {
- /* When `n1' has already been set as a river, there is no
- point in looking at the other neighbors. */
- if(n1!=CLUMPS_RIVER)
- {
- /* For easy reading. */
- nlab=clabel[ nind ];
-
- /* If this neighbor is on a non-processing label, then
- set the first neighbor accordingly. Note that we
- also want the zero valued neighbors (detections if
- working on sky, and sky if working on detection):
- we want rivers between the two domains. */
- n1 = ( nlab
-
- /* nlab is non-zero. */
- ? ( nlab>0
-
- /* Neighbor has a meaningful label, so
- check with any previously found labeled
- neighbors. */
- ? ( n1
- ? ( nlab==n1 ? n1 : CLUMPS_RIVER )
- : nlab )
-
- /* If the dataset has blank values and this
- neighbor is blank, then the pixel should
- be a river. Note that the blank checking
- can be optimized out, so if the input
- doesn't have blank values,
- `nlab==GAL_BLANK_INT32' will never be
- checked. */
- : ( (p->input->flag & GAL_DATA_FLAG_HASBLANK)
- && nlab==GAL_BLANK_INT32
- ? CLUMPS_RIVER : n1 ) )
-
- /* `nlab==0' (the neighbor lies in the other
- domain (sky or detections). To avoid the
- different domains touching, this pixel
- should be a river. */
- : CLUMPS_RIVER );
- }
- });
-
- /* Either assign a new label to this pixel, or give it the one
- of its neighbors. If n1 equals zero, then this is a new
- peak, and a new label should be created. But if n1!=0, it
- is either a river pixel (has more than one labeled neighbor
- and has been set to `CLUMPS_RIVER' before) or all its
- neighbors have the same label. In both such cases, rlab
- should be set to n1.*/
- if(n1) rlab = n1;
- else
- {
- rlab = curlab++;
- if( cltprm->topinds )
- cltprm->topinds[ rlab ]=*a;
- }
-
- /* Put the found label in the pixel. */
- clabel[ *a ] = rlab;
- }
-
- /*********************************************
- For checks and debugging:
- if( *a / dsize[1] >= checkstart[0]
- && *a / dsize[1] < checkstart[0] + checkdsize[0]
- && *a % dsize[1] >= checkstart[1]
- && *a % dsize[1] < checkstart[1] + checkdsize[1] )
- {
- printf("%zu (%zu: %zu, %zu): %u\n", ++extcount, *a,
- (*a%dsize[1])-checkstart[1], (*a/dsize[1])-checkstart[0],
- clabel[*a]);
- crop=gal_data_copy(tile);
- crf=(cr=crop->array)+crop->size;
- do if(*cr==CLUMPS_RIVER) *cr=0; while(++cr<crf);
- gal_fits_img_write(crop, filename, NULL, PROGRAM_NAME);
- gal_data_free(crop);
- }
- **********************************************/
- }
- while(++a<af);
-
- /* Save the total number of clumps. */
- cltprm->numinitclumps=curlab-1;
-
- /* Set all the river pixels to zero. Note that this is only necessary for
- the detected clumps. When finding clumps over the Sky, we will be
- going over the full tile and removing rivers after this function. This
- is because, we set the borders of the tile to a river value and didn't
- include them in the list of indexs. */
- if(cltprm->clprm->sky0_det1)
- {
- af=(a=indexs->array)+indexs->size;
- do if( clabel[*a]==CLUMPS_RIVER ) clabel[*a]=CLUMPS_INIT; while(++a<af);
- }
-
- /*********************************************
- For checks and debugging:
- tile->array=NULL;
- gal_data_free(tile);
- printf("Total number of clumps: %u\n", curlab-1);
- **********************************************/
-
- /* Clean up. */
- free(dinc);
-}
-
-
-
-
-
-
-
-
-
-
@@ -413,13 +65,13 @@ clumps_grow_prepare_initial(struct clumps_thread_params
*cltprm)
{
gal_data_t *indexs=cltprm->indexs;
gal_data_t *input=cltprm->clprm->p->input;
- struct noisechiselparams *p=cltprm->clprm->p;
+ struct segmentparams *p=cltprm->clprm->p;
- size_t *s, *sf, *dsize=input->dsize;
- size_t ndiffuse=0, coord[2], *dindexs;
- double wcoord[2]={0.0f,0.0f}, brightness=0.0f;
- float glimit, *imgss=input->array, *std=p->std->array;
+ double wcoord[3]={0.0f,0.0f,0.0f}, sum=0.0f;
+ size_t ndiffuse=0, coord[3], tcoord[3], *dindexs;
+ size_t *s, *sf, *dsize=input->dsize, ndim=input->ndim;
int32_t *olabel=p->olabel->array, *clabel=p->clabel->array;
+ float glimit, *imgss=input->array, *std=p->std?p->std->array:NULL;
/* Find the flux weighted center (meaningful only for positive valued
@@ -428,35 +80,54 @@ clumps_grow_prepare_initial(struct clumps_thread_params
*cltprm)
do
if( imgss[ *s ] > 0.0f )
{
- brightness += imgss[ *s ];
- wcoord[0] += imgss[ *s ] * (*s/dsize[1]);
- wcoord[1] += imgss[ *s ] * (*s%dsize[1]);
+ gal_dimension_index_to_coord(*s, ndim, dsize, tcoord);
+ sum += imgss[ *s ];
+ wcoord[0] += imgss[ *s ] * tcoord[0];
+ wcoord[1] += imgss[ *s ] * tcoord[1];
+ if(ndim==3)
+ wcoord[1] += imgss[ *s ] * tcoord[2];
}
while(++s<sf);
/* Calculate the center, if no pixels were positive, use the
geometric center (irrespective of flux). */
- if(brightness==0.0f)
+ if(sum==0.0f)
{
sf=(s=indexs->array)+indexs->size;
do
{
- wcoord[0] += *s / dsize[1];
- wcoord[1] += *s % dsize[1];
+ gal_dimension_index_to_coord(*s, ndim, dsize, tcoord);
+ wcoord[0] += tcoord[0];
+ wcoord[1] += tcoord[1];
+ if(ndim==3)
+ wcoord[2] += tcoord[2];
}
while(++s<sf);
- brightness = indexs->size;
+ sum = indexs->size;
}
- /* Convert floatint point coordinates to FITS integers. */
- coord[0] = GAL_DIMENSION_FLT_TO_INT(wcoord[0]);
- coord[1] = GAL_DIMENSION_FLT_TO_INT(wcoord[1]);
+ /* Convert floating point coordinates to integers. */
+ coord[0] = GAL_DIMENSION_FLT_TO_INT(wcoord[0]/sum);
+ coord[1] = GAL_DIMENSION_FLT_TO_INT(wcoord[1]/sum);
+ if(ndim==3)
+ coord[2] = GAL_DIMENSION_FLT_TO_INT(wcoord[2]/sum);
+
+ /* Find the growth limit. Note that the STD may be a value, or a dataset
+ (which may be a full sized image or a tessellation). If its a dataset,
+ `stdval==NAN', and we'll check through the number of elements to see
+ what kind of dataset it is.. */
+ cltprm->std = ( p->std
+ ? ( p->std->size==p->input->size
+ ? std[gal_dimension_coord_to_index(ndim, dsize, coord)]
+ : std[gal_tile_full_id_from_coord(&p->cp.tl, coord)] )
+ : p->stdval );
+ if(p->variance) cltprm->std = sqrt(cltprm->std);
- /* Find the growth limit */
- cltprm->std = std[ gal_tile_full_id_from_coord(&p->cp.tl, coord) ];
+
+ /* From the standard deviation, find the growth limit. */
glimit = p->gthresh * cltprm->std;
@@ -475,7 +146,7 @@ clumps_grow_prepare_initial(struct clumps_thread_params
*cltprm)
do
{
olabel[*s] = clabel[*s];
- if( clabel[*s]==CLUMPS_INIT )
+ if( clabel[*s]==GAL_LABEL_INIT )
if( imgss[*s]>glimit ) dindexs[ ndiffuse++ ] = *s;
}
while(++s<sf);
@@ -523,145 +194,7 @@ clumps_grow_prepare_final(struct clumps_thread_params
*cltprm)
-/* Grow the true clumps over the diffuse regions of a detection. Note that
- unlike before, were river pixels would get a separate label for them
- selves, here, they don't, they just get set back to SEGMENTINIT. This is
- because some of the pixels that lie immediately between two labeled
- regions might not be in the blankinds array (they were below the
- threshold). So we have to find river pixels later on, after the growth
- is done independently.
- This function is going to be used before identifying objects and also
- after it (to completely fill in the diffuse area). The distinguishing
- point between these two steps is the presence of rivers, so you can use
- the `withrivers' argument.
-
-
- Input:
-
- labels: The labels array that must be operated on. The pixels that
- must be "grown" must have the value `CLUMPS_INIT' (negative).
-
- diffuseindexs: The indexs of the pixels that must be grown.
-
- withrivers: as described above.
-
- connectivity: connectivity to define neighbors for growth.
-*/
-void
-clumps_grow(gal_data_t *labels, gal_data_t *diffuseindexs, int withrivers,
- int connectivity)
-{
- int searchngb;
- size_t *diarray=diffuseindexs->array;
- int32_t n1, nlab, *olabel=labels->array;
- size_t *s, *sf, thisround, ndiffuse=diffuseindexs->size;
- size_t *dinc=gal_dimension_increment(labels->ndim, labels->dsize);
-
- /* A small sanity check: */
- if(labels->type!=GAL_TYPE_INT32)
- error(EXIT_FAILURE, 0, "%s: `labels' has to have type of int32_t",
- __func__);
- if(diffuseindexs->type!=GAL_TYPE_SIZE_T)
- error(EXIT_FAILURE, 0, "%s: `diffuseindexs' has to have type of size_t",
- __func__);
-
- /* The basic idea is this: after growing, not all the blank pixels are
- necessarily filled, for example the pixels might belong to two regions
- above the growth threshold. So the pixels in between them (which are
- below the threshold will not ever be able to get a label). Therefore,
- the safest way we can terminate the loop of growing the objects is to
- stop it when the number of pixels left to fill in this round
- (thisround) equals the number of blanks.
-
- To start the loop, we set `thisround' to one more than the number of
- diffuse pixels. Note that it will be corrected immediately after the
- loop has started, it is just important to pass the `while'. */
- thisround=ndiffuse+1;
- while( thisround > ndiffuse )
- {
- /* `thisround' will keep the number of pixels to be inspected in this
- round. `ndiffuse' will count the number of pixels left without a
- label by the end of this round. Since `ndiffuse' comes from the
- previous loop (or outside, for the first round) it has to be saved
- in `thisround' to begin counting a fresh. */
- thisround=ndiffuse;
- ndiffuse=0;
-
- /* Go over all the available indexs. */
- sf=(s=diffuseindexs->array)+diffuseindexs->size;
- do
- {
- /* We'll begin by assuming the nearest neighbor of this pixel has
- no label (has a value of 0). */
- n1=0;
-
- /* Check the very closest neighbors of each pixel (4-connectivity
- in a 2D image). Note that since this macro has multiple loops
- within it, we can't use break. We'll use a variable instead. */
- searchngb=1;
- GAL_DIMENSION_NEIGHBOR_OP(*s, labels->ndim, labels->dsize,
- connectivity, dinc,
- {
- if(searchngb)
- {
- /* For easy reading. */
- nlab = olabel[nind];
-
- /* This neighbor's label is meaningful. */
- if(nlab>0) /* This is a real label. */
- {
- if(n1) /* A prev. neighboring label has been found. */
- {
- if( n1 != nlab ) /* Different label from */
- { /* prevously found neighbor for this pixel. */
- n1=CLUMPS_RIVER;
- searchngb=0;
- }
- }
- else
- { /* This is the first labeld neighbor found. */
- n1=nlab;
-
- /* If we want to completely fill in the region
- (`withrivers==0'), then there is no point in
- looking in other neighbors, the first neighbor
- we find is the one we'll use. */
- if(!withrivers) searchngb=0;
- }
- }
- }
- } );
-
- /* The loop above over neighbors finishes with three
- possibilities:
-
- n1==0 --> No labeled neighbor was found.
- n1==CLUMPS_RIVER --> Connecting two labeled regions.
- n1>0 --> Only has one neighbouring label.
-
- The first one means that no neighbors were found and this
- pixel should be kept for the next loop (we'll be growing the
- objects pixel-layer by pixel-layer). In the other two cases,
- we just need to write in the value of `n1'. */
- if(n1)
- {
- olabel[*s]=n1;
- if(withrivers && n1==CLUMPS_RIVER) /* To keep rivers in */
- diarray[ ndiffuse++ ] = *s; /* the diffuse list. */
- }
- else
- diarray[ ndiffuse++ ] = *s;
-
- /* Correct the size of the `diffuseindexs' dataset. */
- diffuseindexs->size = diffuseindexs->dsize[0] = ndiffuse;
- }
- while(++s<sf);
- }
-
- /* Clean up. */
- free(dinc);
-}
@@ -708,15 +241,16 @@ enum infocols
static void
clumps_get_raw_info(struct clumps_thread_params *cltprm)
{
- struct noisechiselparams *p=cltprm->clprm->p;
+ struct segmentparams *p=cltprm->clprm->p;
size_t ndim=p->input->ndim, *dsize=p->input->dsize;
+ size_t i, *a, *af, ii, coord[3];
double *row, *info=cltprm->info->array;
- size_t i, *a, *af, ii, fullid, coord[3];
size_t nngb=gal_dimension_num_neighbors(ndim);
- float *arr=p->input->array, *std=p->std->array;
+ struct gal_tile_two_layer_params *tl=&p->cp.tl;
size_t *dinc=gal_dimension_increment(ndim, dsize);
int32_t lab, nlab, *ngblabs, *clabel=p->clabel->array;
+ float *arr=p->input->array, *std=p->std?p->std->array:NULL;
/* Allocate the array to keep the neighbor labels of river pixels. */
ngblabs=gal_data_malloc_array(GAL_TYPE_INT32, nngb, __func__, "ngblabs");
@@ -792,7 +326,7 @@ clumps_get_raw_info(struct clumps_thread_params *cltprm)
for(lab=1; lab<=cltprm->numinitclumps; ++lab)
{
row = &info [ lab * INFO_NCOLS ];
- if ( row[INFO_INAREA] > p->segsnminarea )
+ if ( row[INFO_INAREA] > p->snminarea )
{
/* Especially over the undetected regions, it might happen that
none of the pixels were positive. In that case, set the total
@@ -807,8 +341,14 @@ clumps_get_raw_info(struct clumps_thread_params *cltprm)
coord[2]=GAL_DIMENSION_FLT_TO_INT(row[INFO_Z]/row[INFO_SFF]);
/* Find the corresponding standard deviation. */
- fullid=gal_tile_full_id_from_coord(&p->cp.tl, coord);
- row[INFO_INSTD] = std[fullid];
+ row[INFO_INSTD]=( p->std
+ ? ( p->std->size==p->input->size
+ ? std[gal_dimension_coord_to_index(ndim,
+ dsize, coord)]
+ : std[gal_tile_full_id_from_coord(tl,
+ coord)] )
+ : p->stdval );
+ if(p->variance) row[INFO_INSTD] = sqrt(row[INFO_INSTD]);
/* For a check
printf("---------\n");
@@ -834,7 +374,7 @@ clumps_get_raw_info(struct clumps_thread_params *cltprm)
void
clumps_make_sn_table(struct clumps_thread_params *cltprm)
{
- struct noisechiselparams *p=cltprm->clprm->p;
+ struct segmentparams *p=cltprm->clprm->p;
size_t tablen=cltprm->numinitclumps+1;
float *snarr;
@@ -900,13 +440,10 @@ clumps_make_sn_table(struct clumps_thread_params *cltprm)
noise cases) or the area is smaller than the minimum area to
calculate signal-to-noise, then set the S/N of this segment to
zero. */
- if( I>O && Ni>p->segsnminarea )
+ if( I>O && Ni>p->snminarea )
{
- /* Here we have done sky subtraction once. However, if the sky
- was already subtracted (informed by the user), then the
- varience should be multiplied by 2. */
- var = ( (p->skysubtracted ? 2.0f : 1.0f)
- * row[INFO_INSTD] * row[INFO_INSTD] );
+ /* For easy reading, define `var' for variance. */
+ var = row[INFO_INSTD] * row[INFO_INSTD];
/* Calculate the Signal to noise ratio, if we are on the noise
regions, we don't care about the IDs of the clumps anymore, so
@@ -955,7 +492,7 @@ clumps_correct_sky_labels_for_check(struct
clumps_thread_params *cltprm,
gal_data_t *newinds;
int32_t *ninds, curlab, *l, *lf;
size_t len=cltprm->numinitclumps+1;
- struct noisechiselparams *p=cltprm->clprm->p;
+ struct segmentparams *p=cltprm->clprm->p;
/* If there are no clumps in this tile, then this function can be
ignored. */
@@ -987,9 +524,10 @@ clumps_correct_sky_labels_for_check(struct
clumps_thread_params *cltprm,
if(p->cp.numthreads>1) pthread_mutex_unlock(&cltprm->clprm->labmutex);
- /* Initialize the newinds array to CLUMPS_INIT (which be used as a new
+ /* Initialize the newinds array to GAL_LABEL_INIT (which be used as a new
label for all the clumps that must be removed. */
- lf = (l=newinds->array) + newinds->size; do *l++=CLUMPS_INIT; while(l<lf);
+ lf = (l=newinds->array) + newinds->size;
+ do *l++=GAL_LABEL_INIT; while(l<lf);
/* The new indexs array has been initialized to zero. So we just need to
@@ -1017,7 +555,7 @@ clumps_find_make_sn_table(void *in_prm)
{
struct gal_threads_params *tprm=(struct gal_threads_params *)in_prm;
struct clumps_params *clprm=(struct clumps_params *)(tprm->params);
- struct noisechiselparams *p=clprm->p;
+ struct segmentparams *p=clprm->p;
size_t ndim=p->input->ndim, *dsize=p->input->dsize;
void *tarray;
@@ -1056,7 +594,7 @@ clumps_find_make_sn_table(void *in_prm)
/* Get the number of usable elements in this tile (note that tiles
can have blank pixels), so we can't simply use `tile->size'. */
- if(tile->flag & GAL_DATA_FLAG_HASBLANK)
+ if(p->input->flag & GAL_DATA_FLAG_HASBLANK)
{
tmp=gal_statistics_number(tile);
num=*((size_t *)(tmp->array));
@@ -1134,7 +672,7 @@ clumps_find_make_sn_table(void *in_prm)
/* If this pixel is on the edge, then it should be a
river. */
if(pixonedge)
- *(int32_t *)i=CLUMPS_RIVER;
+ *(int32_t *)i=GAL_LABEL_RIVER;
/* This pixel is not on the edge, check if it had a value of
`0' in the binary image (is not detected) then add it to
@@ -1171,13 +709,15 @@ clumps_find_make_sn_table(void *in_prm)
/* Generate the clumps over this region. */
- clumps_oversegment(&cltprm);
+ cltprm.numinitclumps=gal_label_oversegment(p->conv, cltprm.indexs,
+ p->clabel,
+ cltprm.topinds);
- /* Set all river pixels to CLUMPS_INIT (to be distinguishable
+ /* Set all river pixels to GAL_LABEL_INIT (to be distinguishable
from the detected regions). */
GAL_TILE_PO_OISET( int32_t, int, tile, NULL, 0, 1,
- {if(*i==CLUMPS_RIVER) *i=CLUMPS_INIT;} );
+ {if(*i==GAL_LABEL_RIVER) *i=GAL_LABEL_INIT;} );
/* For a check, the step variable will be set. */
@@ -1218,6 +758,59 @@ clumps_find_make_sn_table(void *in_prm)
+/* Write the S/N table. */
+static void
+clumps_write_sn_table(struct segmentparams *p, gal_data_t *insn,
+ gal_data_t *inind, char *filename,
+ gal_list_str_t *comments)
+{
+ gal_data_t *sn, *ind, *cols;
+
+ /* Remove all blank elements. The index and sn values must have the same
+ set of blank elements, but checking on the integer array is faster. */
+ if( gal_blank_present(inind, 1) )
+ {
+ /* Remove blank elements. */
+ ind=gal_data_copy(inind);
+ sn=gal_data_copy(insn);
+ gal_blank_remove(ind);
+ gal_blank_remove(sn);
+
+ /* A small sanity check. */
+ if(ind->size==0 || sn->size==0)
+ error(EXIT_FAILURE, 0, "%s: a bug! Please contact us at %s to fix "
+ "the problem. For some reason, all the elements in `ind' or "
+ "`sn' are blank", __func__, PACKAGE_BUGREPORT);
+ }
+ else
+ {
+ sn = insn;
+ ind = inind;
+ }
+
+ /* Set the columns. */
+ cols = ind;
+ cols->next = sn;
+
+
+ /* Prepare the comments. */
+ gal_table_comments_add_intro(&comments, PROGRAM_STRING, &p->rawtime);
+
+
+ /* write the table. */
+ gal_checkset_writable_remove(filename, 0, 1);
+ gal_table_write(cols, comments, p->cp.tableformat, filename, "SN");
+
+
+ /* Clean up (if necessary). */
+ if(sn!=insn) gal_data_free(sn);
+ if(ind==inind) ind->next=NULL; else gal_data_free(ind);
+}
+
+
+
+
+
/* Find the true clump signal to noise value from the clumps in the sky
region.
@@ -1234,7 +827,7 @@ clumps_find_make_sn_table(void *in_prm)
concatenate all the S/N values into one array and send it to the main
findsnthresh function in thresh.c. */
void
-clumps_true_find_sn_thresh(struct noisechiselparams *p)
+clumps_true_find_sn_thresh(struct segmentparams *p)
{
char *msg;
struct timeval t1;
@@ -1253,7 +846,7 @@ clumps_true_find_sn_thresh(struct noisechiselparams *p)
clprm.p=p;
clprm.sky0_det1=0;
clprm.sn=gal_data_array_calloc(p->ltl.tottiles);
- clprm.snind = ( p->checksegmentation || p->checkclumpsn
+ clprm.snind = ( p->checksegmentation || p->checksn
? gal_data_array_calloc(p->ltl.tottiles) : NULL );
@@ -1263,7 +856,7 @@ clumps_true_find_sn_thresh(struct noisechiselparams *p)
label. Since `p->numclumps' is not used yet, we will use it here. When
multiple threads are used, we will need a mutex to make sure that only
one thread can change this central variable at every one moment. */
- if(p->checksegmentation || p->checkclumpsn)
+ if(p->checksegmentation || p->checksn)
{
p->numclumps=0;
if( p->cp.numthreads > 1 ) pthread_mutex_init(&clprm.labmutex, NULL);
@@ -1330,7 +923,7 @@ clumps_true_find_sn_thresh(struct noisechiselparams *p)
/* Destroy the mutex if it was initialized. */
- if( p->cp.numthreads>1 && (p->checksegmentation || p->checkclumpsn) )
+ if( p->cp.numthreads>1 && (p->checksegmentation || p->checksn) )
pthread_mutex_destroy(&clprm.labmutex);
@@ -1339,18 +932,42 @@ clumps_true_find_sn_thresh(struct noisechiselparams *p)
if(clprm.sn[i].ndim) /* Only on tiles were an S/N was calculated. */
numsn+=clprm.sn[i].size;
if( numsn < p->minnumfalse )
- error(EXIT_FAILURE, 0, "only %zu clumps could be identified in the "
- "undetected regions. This is less than %zu (value to "
- "`--minnumfalse' option). Please either decrease this value or "
- "other options to change prior processing steps", numsn,
- p->minnumfalse);
+ error(EXIT_FAILURE, 0, "%zu usable clumps found in the undetected "
+ "regions. This is smaller than the requested minimum number of "
+ "false/reference clumps (%zu, value to the `--minnumfalse' "
+ "option).\n\n"
+ "There are several ways to address the problem. The best and most "
+ "highly recommended is to use a larger input if possible (when the "
+ "input is a crop from a larger dataset). If that is not the case, "
+ "or it doesn't solve the problem, you need to loosen the "
+ "parameters (and therefore cause more scatter/bias in the final "
+ "result). Thus don't loosen them too much. Recall that you can "
+ "see all the option values to Gnuastro's programs by appending "
+ "`-P' to the end of your command.\n\n"
+ " * Slightly decrease `--largetilesize' to have more tiles.\n"
+ " * Decrease `--minskyfrac' (currently %g) to look into more "
+ "tiles.\n"
+ " * Slightly decrease `--snminarea' (currently %zu) to "
+ "measure more clumps.\n"
+ " * If Segment already works on a dataset with similar noise "
+ "properties, you can directly pass the 'true' clump "
+ "signal-to-noise ratio found there to `--clumpsnthresh' and "
+ "avoid having to study the undetected regions any more.\n\n"
+ "Append your previous command with `--checksegmentation' to see "
+ "the steps and get a better feeling of the cause/solution. Note "
+ "that the output is a multi-extension FITS file).\n\n"
+ "To better understand the segmentation process and options, "
+ "please run the following command (press `SPACE'/arrow-keys to "
+ "navigate and `Q' to return back to the command-line):\n\n"
+ " $ info gnuastro \"Segmentation options\"\n",
+ numsn, p->minnumfalse, p->minskyfrac, p->snminarea);
/* Allocate the space to keep all the S/N values. */
sn=gal_data_alloc(NULL, GAL_TYPE_FLOAT32, 1, &numsn, NULL, 0,
p->cp.minmapsize, "CLUMP_S/N", "ratio",
"Signal-to-noise ratio");
- snind = ( p->checkclumpsn
+ snind = ( p->checksn
? gal_data_alloc(NULL, GAL_TYPE_INT32, 1, &numsn, NULL, 0,
p->cp.minmapsize, "CLUMP_ID", "counter",
"Unique ID for this clump.")
@@ -1377,7 +994,7 @@ clumps_true_find_sn_thresh(struct noisechiselparams *p)
/* If the user wanted to see the S/N table, then save it. */
- if(p->checkclumpsn)
+ if(p->checksn)
{
/* Make the comments, then write the table and free the comments. */
if(p->cp.numthreads>1)
@@ -1387,18 +1004,18 @@ clumps_true_find_sn_thresh(struct noisechiselparams *p)
"output with `--checksegmentation'.", 1);
gal_list_str_add(&comments, "S/N of clumps over undetected regions.",
1);
- threshold_write_sn_table(p, sn, snind, p->clumpsn_s_name, comments);
+ clumps_write_sn_table(p, sn, snind, p->clumpsn_s_name, comments);
gal_list_str_free(comments, 1);
}
/* Find the desired quantile from the full S/N distribution. */
- quant = gal_statistics_quantile(sn, p->segquant, 1);
+ quant = gal_statistics_quantile(sn, p->snquant, 1);
p->clumpsnthresh = *((float *)(quant->array));
if(!p->cp.quiet)
{
if( asprintf(&msg, "Clump S/N: %.2f (%.3f quant of %zu).",
- p->clumpsnthresh, p->segquant, sn->size)<0 )
+ p->clumpsnthresh, p->snquant, sn->size)<0 )
error(EXIT_FAILURE, 0, "%s: asprintf allocation", __func__);
gal_timing_report(&t1, msg, 2);
free(msg);
@@ -1437,7 +1054,7 @@ clumps_true_find_sn_thresh(struct noisechiselparams *p)
(where each element is a dataset containing the respective label's
indexs). */
gal_data_t *
-clumps_det_label_indexs(struct noisechiselparams *p)
+clumps_det_label_indexs(struct segmentparams *p)
{
size_t i, *areas;
int32_t *a, *l, *lf;
@@ -1491,7 +1108,7 @@ clumps_det_label_indexs(struct noisechiselparams *p)
void
clumps_det_keep_true_relabel(struct clumps_thread_params *cltprm)
{
- struct noisechiselparams *p=cltprm->clprm->p;
+ struct segmentparams *p=cltprm->clprm->p;
size_t ndim=p->input->ndim, *dsize=p->input->dsize;
int istouching;
@@ -1510,10 +1127,10 @@ clumps_det_keep_true_relabel(struct
clumps_thread_params *cltprm)
"newlabs");
dinc=gal_dimension_increment(ndim, dsize);
- /* Initialize the new labels with CLUMPS_INIT (so the diffuse area
+ /* Initialize the new labels with GAL_LABEL_INIT (so the diffuse area
can be distinguished from the clumps). */
lf=(l=newlabs)+cltprm->numinitclumps+1;
- do *l++=CLUMPS_INIT; while(l<lf);
+ do *l++=GAL_LABEL_INIT; while(l<lf);
/* Set the new labels. Here we will also be removing clumps with a peak
that touches a river pixel. */
@@ -1543,10 +1160,10 @@ clumps_det_keep_true_relabel(struct
clumps_thread_params *cltprm)
}
}
- /* Correct the clump labels. Note that the non-clumpy regions over the
- detections (rivers) have already been initialized to CLUMPS_INIT
- (which is negative). So we'll just need to correct the ones with a
- value larger than 0. */
+ /* Correct the clump labels. Note that the non-clumpy regions over
+ the detections (rivers) have already been initialized to
+ GAL_LABEL_INIT (which is negative). So we'll just need to correct
+ the ones with a value larger than 0. */
sf=(s=cltprm->indexs->array)+cltprm->indexs->size;
do if(clabel[*s]>0) clabel[*s] = newlabs[ clabel[*s] ]; while(++s<sf);
diff --git a/bin/noisechisel/clumps.h b/bin/segment/clumps.h
similarity index 86%
rename from bin/noisechisel/clumps.h
rename to bin/segment/clumps.h
index 531f6c3..9799c45 100644
--- a/bin/noisechisel/clumps.h
+++ b/bin/segment/clumps.h
@@ -1,6 +1,6 @@
/*********************************************************************
-NoiseChisel - Detect and segment signal in a noisy dataset.
-NoiseChisel is part of GNU Astronomy Utilities (Gnuastro) package.
+Segment - Segment initial labels based on signal structure.
+Segment is part of GNU Astronomy Utilities (Gnuastro) package.
Original author:
Mohammad Akhlaghi <mohammad@akhlaghi.org>
@@ -24,11 +24,6 @@ along with Gnuastro. If not, see
<http://www.gnu.org/licenses/>.
#define CLUMPS_H
-/* Constants for the clump over-segmentation. */
-#define CLUMPS_INIT -1
-#define CLUMPS_RIVER -2
-#define CLUMPS_TMPCHECK -3
-
/* Parameters for all threads. */
struct clumps_params
@@ -36,7 +31,7 @@ struct clumps_params
/* General */
int step; /* Counter if we want to check steps. */
int sky0_det1; /* If working on the Sky or Detections. */
- struct noisechiselparams *p; /* Pointer to main NoiseChisel parameters. */
+ struct segmentparams *p; /* Pointer to main Segment parameters. */
pthread_mutex_t labmutex; /* Mutex to change the total numbers. */
/* For Sky region. */
@@ -68,10 +63,6 @@ struct clumps_thread_params
struct clumps_params *clprm; /* Pointer to main structure. */
};
-
-void
-clumps_oversegment(struct clumps_thread_params *cltprm);
-
void
clumps_grow_prepare_initial(struct clumps_thread_params *cltprm);
@@ -83,13 +74,13 @@ clumps_grow(gal_data_t *labels, gal_data_t *diffuseindexs,
int withrivers,
int connectivity);
void
-clumps_true_find_sn_thresh(struct noisechiselparams *p);
+clumps_true_find_sn_thresh(struct segmentparams *p);
void
clumps_make_sn_table(struct clumps_thread_params *cltprm);
gal_data_t *
-clumps_det_label_indexs(struct noisechiselparams *p);
+clumps_det_label_indexs(struct segmentparams *p);
void
clumps_det_keep_true_relabel(struct clumps_thread_params *cltprm);
diff --git a/bin/noisechisel/main.c b/bin/segment/main.c
similarity index 79%
copy from bin/noisechisel/main.c
copy to bin/segment/main.c
index c49c91b..34254a3 100644
--- a/bin/noisechisel/main.c
+++ b/bin/segment/main.c
@@ -1,11 +1,11 @@
/*********************************************************************
-NoiseChisel - Detect and segment signal in a noisy dataset.
-NoiseChisel is part of GNU Astronomy Utilities (Gnuastro) package.
+Segment - Segment initial labels based on signal structure.
+Segment is part of GNU Astronomy Utilities (Gnuastro) package.
Original author:
Mohammad Akhlaghi <mohammad@akhlaghi.org>
Contributing author(s):
-Copyright (C) 2015-2018, Free Software Foundation, Inc.
+Copyright (C) 2018, Free Software Foundation, Inc.
Gnuastro is free software: you can redistribute it and/or modify it
under the terms of the GNU General Public License as published by the
@@ -30,7 +30,7 @@ along with Gnuastro. If not, see
<http://www.gnu.org/licenses/>.
#include "main.h"
#include "ui.h"
-#include "noisechisel.h"
+#include "segment.h"
/* Main function */
@@ -38,17 +38,17 @@ int
main (int argc, char *argv[])
{
struct timeval t1;
- struct noisechiselparams p={{{0},0},{0},0};
+ struct segmentparams p={{{0},0},{0},0};
- /* Set they starting time. */
+ /* Set the starting time. */
time(&p.rawtime);
gettimeofday(&t1, NULL);
/* Read the input parameters. */
ui_read_check_inputs_setup(argc, argv, &p);
- /* Run MakeProfiles */
- noisechisel(&p);
+ /* Run Segment */
+ segment(&p);
/* Free all non-freed allocations. */
ui_free_report(&p, &t1);
diff --git a/bin/segment/main.h b/bin/segment/main.h
new file mode 100644
index 0000000..62f3102
--- /dev/null
+++ b/bin/segment/main.h
@@ -0,0 +1,110 @@
+/*********************************************************************
+Segment - Segment initial labels based on signal structure.
+Segment is part of GNU Astronomy Utilities (Gnuastro) package.
+
+Original author:
+ Mohammad Akhlaghi <mohammad@akhlaghi.org>
+Contributing author(s):
+Copyright (C) 2018, Free Software Foundation, Inc.
+
+Gnuastro is free software: you can redistribute it and/or modify it
+under the terms of the GNU General Public License as published by the
+Free Software Foundation, either version 3 of the License, or (at your
+option) any later version.
+
+Gnuastro is distributed in the hope that it will be useful, but
+WITHOUT ANY WARRANTY; without even the implied warranty of
+MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the GNU
+General Public License for more details.
+
+You should have received a copy of the GNU General Public License
+along with Gnuastro. If not, see <http://www.gnu.org/licenses/>.
+**********************************************************************/
+#ifndef MAIN_H
+#define MAIN_H
+
+/* Include necessary headers */
+#include <gnuastro/data.h>
+
+#include <gnuastro-internal/options.h>
+
+/* Progarm names. */
+#define PROGRAM_NAME "Segment" /* Program full name. */
+#define PROGRAM_EXEC "astsegment" /* Program executable name. */
+#define PROGRAM_STRING PROGRAM_NAME" (" PACKAGE_NAME ") " PACKAGE_VERSION
+
+
+/* Macros */
+#define DETECTION_ALL "all"
+
+
+
+
+
+
+/* Main program parameters structure */
+struct segmentparams
+{
+ /* From command-line */
+ struct gal_options_common_params cp; /* Common parameters. */
+ struct gal_tile_two_layer_params ltl; /* Large tessellation. */
+ char *inputname; /* Input filename. */
+ char *kernelname; /* Input kernel filename. */
+ char *khdu; /* Kernel HDU. */
+ char *convolvedname; /* Convolved image (to avoid convolution).*/
+ char *chdu; /* HDU of convolved image. */
+ char *detectionname; /* Detection image file name. */
+ char *dhdu; /* Detection image file name. */
+ char *skyname; /* Filename of Sky image. */
+ char *skyhdu; /* Filename of Sky image. */
+ char *stdname; /* File name of Standard deviation image. */
+ char *stdhdu; /* HDU of Stanard deviation image. */
+ uint8_t variance; /* The input STD is actually variance. */
+ uint8_t rawoutput; /* Output only object and clump labels. */
+
+ float minskyfrac; /* Undetected area min. frac. in tile. */
+ size_t snminarea; /* Minimum area for segmentation. */
+ uint8_t checksn; /* Save the clump S/N values to a file. */
+ size_t minnumfalse; /* Min No. of det/seg for true quantile. */
+ float snquant; /* Quantile of clumps in sky for true S/N.*/
+ uint8_t keepmaxnearriver; /* Keep clumps with a peak near a river. */
+ float clumpsnthresh; /* Clump S/N threshold. */
+ float gthresh; /* Multiple of STD to stop growing clumps.*/
+ size_t minriverlength; /* Min, len of good grown clump rivers. */
+ float objbordersn; /* Minimum S/N for grown clumps to be one.*/
+ uint8_t grownclumps; /* Save grown clumps instead of original. */
+ uint8_t continueaftercheck; /* Don't abort after the check steps. */
+ uint8_t checksegmentation; /* Save the segmentation steps in file. */
+
+ /* Internal. */
+ char *clumpsn_s_name; /* Sky clump S/N name. */
+ char *clumpsn_d_name; /* Detection clumps S/N name. */
+ char *segmentationname; /* Name of segmentation steps file. */
+
+ gal_data_t *input; /* Input dataset. */
+ gal_data_t *kernel; /* Given kernel for convolution. */
+ gal_data_t *conv; /* Convolved dataset. */
+ gal_data_t *binary; /* For binary operations. */
+ gal_data_t *olabel; /* Object labels. */
+ gal_data_t *clabel; /* Clumps labels. */
+ gal_data_t *std; /* STD of undetected pixels, per tile. */
+ float stdval; /* Single value to use for std deviation. */
+ float skyval; /* Single value to use for Sky. */
+
+ float cpscorr; /* Counts/second correction. */
+ size_t numdetections; /* Number of final detections. */
+ size_t numclumps; /* Number of true clumps. */
+ size_t numobjects; /* Number of objects. */
+
+ char *useddetectionname; /* Name of file USED for detection image. */
+ char *usedstdname; /* Name of file USED for sky STD image. */
+
+ float medstd; /* For output STD image: median STD. */
+ float minstd; /* For output STD image: median STD. */
+ float maxstd; /* For output STD image: median STD. */
+
+ /* Output: */
+ time_t rawtime; /* Starting time of the program. */
+};
+
+#endif
diff --git a/bin/noisechisel/segmentation.c b/bin/segment/segment.c
similarity index 76%
rename from bin/noisechisel/segmentation.c
rename to bin/segment/segment.c
index c54453b..9fc1f5c 100644
--- a/bin/noisechisel/segmentation.c
+++ b/bin/segment/segment.c
@@ -1,6 +1,6 @@
/*********************************************************************
-NoiseChisel - Detect and segment signal in a noisy dataset.
-NoiseChisel is part of GNU Astronomy Utilities (Gnuastro) package.
+Segment - Segment initial labels based on signal structure.
+Segment is part of GNU Astronomy Utilities (Gnuastro) package.
Original author:
Mohammad Akhlaghi <mohammad@akhlaghi.org>
@@ -28,11 +28,15 @@ along with Gnuastro. If not, see
<http://www.gnu.org/licenses/>.
#include <stdlib.h>
#include <string.h>
+#include <gnuastro/wcs.h>
#include <gnuastro/fits.h>
#include <gnuastro/blank.h>
+#include <gnuastro/label.h>
#include <gnuastro/binary.h>
#include <gnuastro/threads.h>
+#include <gnuastro/convolve.h>
#include <gnuastro/dimension.h>
+#include <gnuastro/statistics.h>
#include <gnuastro-internal/timing.h>
#include <gnuastro-internal/checkset.h>
@@ -41,7 +45,134 @@ along with Gnuastro. If not, see
<http://www.gnu.org/licenses/>.
#include "ui.h"
#include "clumps.h"
-#include "segmentation.h"
+#include "segment.h"
+
+
+
+
+
+
+
+
+
+
+/***********************************************************************/
+/***************** Preparations *****************/
+/***********************************************************************/
+static void
+segment_convolve(struct segmentparams *p)
+{
+ struct timeval t1;
+ struct gal_tile_two_layer_params *tl=&p->cp.tl;
+
+ /* Convovle with sharper kernel. */
+ if(p->conv==NULL)
+ {
+ /* Do the convolution if a kernel was requested. */
+ if(p->kernel)
+ {
+ /* Make the convolved image. */
+ if(!p->cp.quiet) gettimeofday(&t1, NULL);
+ p->conv = gal_convolve_spatial(tl->tiles, p->kernel,
+ p->cp.numthreads, 1, tl->workoverch);
+
+ /* Report and write check images if necessary. */
+ if(!p->cp.quiet)
+ gal_timing_report(&t1, "Convolved with given kernel.", 1);
+ }
+ else
+ p->conv=p->input;
+ }
+
+ /* Make necessary corrections to the convolved array. */
+ if(p->conv!=p->input)
+ {
+ /* Set the flags (most importantly, the blank flags). */
+ p->conv->flag = p->input->flag;
+
+ /* Set the name. */
+ if(p->conv->name) free(p->conv->name);
+ gal_checkset_allocate_copy("CONVOLVED", &p->conv->name);
+ }
+}
+
+
+
+
+
+static void
+segment_initialize(struct segmentparams *p)
+{
+ uint8_t *b;
+ float *f, minv;
+ gal_data_t *min;
+ int32_t *o, *c, *cf;
+
+ /* Allocate the clump labels image and the binary image. */
+ p->clabel=gal_data_alloc(NULL, p->olabel->type, p->olabel->ndim,
+ p->olabel->dsize, p->olabel->wcs, 1,
+ p->cp.minmapsize, NULL, NULL, NULL);
+ p->binary=gal_data_alloc(NULL, GAL_TYPE_UINT8, p->olabel->ndim,
+ p->olabel->dsize, p->olabel->wcs, 1,
+ p->cp.minmapsize, NULL, NULL, NULL);
+ p->clabel->flag=p->input->flag;
+ p->binary->wcs=gal_wcs_copy(p->input->wcs);
+ p->clabel->wcs=gal_wcs_copy(p->input->wcs);
+
+
+ /* Prepare the `binary', `clabel' and `olabel' arrays. */
+ b=p->binary->array;
+ o=p->olabel->array;
+ f=p->input->array; cf=(c=p->clabel->array)+p->clabel->size;
+ do
+ {
+ if(isnan(*f++)) *o = *c = GAL_BLANK_INT32;
+ else
+ {
+ /* Initialize the binary array. */
+ *b = *o > 0;
+
+ /* A small sanity check. */
+ if(*o<0)
+ error(EXIT_FAILURE, 0, "%s (hdu: %s) has negative value(s). "
+ "Each non-zero pixel in this image must be positive (a "
+ "counter, counting from 1).", p->useddetectionname,
+ p->dhdu);
+ }
+ ++o;
+ ++b;
+ }
+ while(++c<cf);
+
+
+ /* If the (minimum) standard deviation is less than 1, then the units of
+ the input are in units of counts/time. As described in the NoiseChisel
+ paper, we need to correct the S/N equation later. */
+ if(p->std)
+ {
+ min=gal_statistics_minimum(p->std);
+ minv=*(float *)(min->array);
+ gal_data_free(min);
+ }
+ else minv=p->stdval;
+ if(p->variance) minv=sqrt(minv);
+ p->cpscorr = minv>1 ? 1.0 : minv;
+}
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
@@ -55,7 +186,7 @@ along with Gnuastro. If not, see
<http://www.gnu.org/licenses/>.
composed of multiple objects). So the labels within each detection start
from 1.*/
static void
-segmentation_relab_noseg(struct clumps_thread_params *cltprm)
+segment_relab_noseg(struct clumps_thread_params *cltprm)
{
int32_t *olabel=cltprm->clprm->p->olabel->array;
size_t *s=cltprm->indexs->array, *sf=s+cltprm->indexs->size;
@@ -76,10 +207,10 @@ segmentation_relab_noseg(struct clumps_thread_params
*cltprm)
a*numobjs+b or b*numobjs+a and get the answer. Since the number of
objects in a given region will not be too high, this is efficient. */
static void
-segmentation_relab_to_objects(struct clumps_thread_params *cltprm)
+segment_relab_to_objects(struct clumps_thread_params *cltprm)
{
size_t amwidth=cltprm->numtrueclumps+1;
- struct noisechiselparams *p=cltprm->clprm->p;
+ struct segmentparams *p=cltprm->clprm->p;
size_t ndim=p->input->ndim, *dsize=p->input->dsize;
size_t mdsize[2]={amwidth, amwidth};
@@ -91,13 +222,13 @@ segmentation_relab_to_objects(struct clumps_thread_params
*cltprm)
NULL, 1, p->cp.minmapsize, NULL,
NULL, NULL);
float *imgss=p->input->array;
+ double var=cltprm->std*cltprm->std;
uint8_t *adjacency=adjacency_d->array;
size_t nngb=gal_dimension_num_neighbors(ndim);
int32_t *clumptoobj, *olabel=p->olabel->array;
size_t *dinc=gal_dimension_increment(ndim, dsize);
size_t *s, *sf, i, j, ii, rpnum, *nums=nums_d->array;
double ave, rpsum, c=sqrt(1/p->cpscorr), *sums=sums_d->array;
- double err=cltprm->std*cltprm->std*(p->skysubtracted?1.0f:2.0f);
int32_t *ngblabs=gal_data_malloc_array(GAL_TYPE_UINT32, nngb, __func__,
"ngblabs");
@@ -114,7 +245,7 @@ segmentation_relab_to_objects(struct clumps_thread_params
*cltprm)
do
/* We only want to work on pixels that have already been identified
as touching more than one label: river pixels. */
- if( olabel[ *s ]==CLUMPS_RIVER )
+ if( olabel[ *s ]==GAL_LABEL_RIVER )
{
/* Initialize the values. */
i=ii=0;
@@ -188,7 +319,7 @@ segmentation_relab_to_objects(struct clumps_thread_params
*cltprm)
acceptable, and we put no area criteria here, because
the fact that a river exists between two clumps is
important. */
- if( ave>0.0f && ( c * ave / sqrt(ave+err) ) > p->objbordersn )
+ if( ave>0.0f && ( c * ave / sqrt(ave+var) ) > p->objbordersn )
{
adjacency[ii]=1; /* We want to set both sides of the */
adjacency[ j * amwidth + i ] = 1; /* Symmetric matrix. */
@@ -213,7 +344,7 @@ segmentation_relab_to_objects(struct clumps_thread_params
*cltprm)
ave=sums[ii]/nums[ii];
printf(" ...%zu: N:%-4zu S:%-10.2f S/N: %-10.2f "
"--> %u\n", j, nums[ii], sums[ii],
- c*ave/sqrt(ave+err), adjacency[ii]);
+ c*ave/sqrt(ave+var), adjacency[ii]);
}
}
printf("\n");
@@ -283,7 +414,7 @@ segmentation_relab_to_objects(struct clumps_thread_params
*cltprm)
this function, we want to correct the clump labels so the clump IDs in
each object start from 1 and are contiguous. */
static void
-segmentation_relab_clumps_in_objects(struct clumps_thread_params *cltprm)
+segment_relab_clumps_in_objects(struct clumps_thread_params *cltprm)
{
size_t numobjects=cltprm->numobjects, numtrueclumps=cltprm->numtrueclumps;
@@ -319,7 +450,7 @@ segmentation_relab_clumps_in_objects(struct
clumps_thread_params *cltprm)
instance, this function will use a mutex to limit the reading and
writing to the variable keeping the total number of objects counter. */
static void
-segmentation_relab_overall(struct clumps_thread_params *cltprm)
+segment_relab_overall(struct clumps_thread_params *cltprm)
{
struct clumps_params *clprm=cltprm->clprm;
int32_t startinglab, *olabel=clprm->p->olabel->array;
@@ -369,11 +500,11 @@ segmentation_relab_overall(struct clumps_thread_params
*cltprm)
/***********************************************************************/
/* Find the true clumps over each detection. */
static void *
-segmentation_on_threads(void *in_prm)
+segment_on_threads(void *in_prm)
{
struct gal_threads_params *tprm=(struct gal_threads_params *)in_prm;
struct clumps_params *clprm=(struct clumps_params *)(tprm->params);
- struct noisechiselparams *p=clprm->p;
+ struct segmentparams *p=clprm->p;
size_t i, *s, *sf;
gal_data_t *topinds;
@@ -410,7 +541,16 @@ segmentation_on_threads(void *in_prm)
/* Find the clumps over this region. */
- clumps_oversegment(&cltprm);
+ cltprm.numinitclumps=gal_label_oversegment(p->conv, cltprm.indexs,
+ p->clabel, cltprm.topinds);
+
+
+ /* Set all the river pixels to zero (we don't need them any more in
+ the clumps image). */
+ sf=(s=cltprm.indexs->array) + cltprm.indexs->size;
+ do
+ if( clabel[*s]==GAL_LABEL_RIVER ) clabel[*s]=GAL_LABEL_INIT;
+ while(++s<sf);
/* Make the clump S/N table. This table is made before (possibly)
@@ -430,7 +570,7 @@ segmentation_on_threads(void *in_prm)
/* If the user wanted to check the segmentation steps or the clump
S/N values in a table, then we have to stop the process at this
point. */
- if(clprm->step==1 || p->checkclumpsn)
+ if(clprm->step==1 || p->checksn)
{ gal_data_free(topinds); continue; }
/* Only keep true clumps. */
@@ -450,7 +590,7 @@ segmentation_on_threads(void *in_prm)
{
/* Set the basics. */
cltprm.numobjects=1;
- segmentation_relab_noseg(&cltprm);
+ segment_relab_noseg(&cltprm);
/* If the user wanted a check image, this object doesn't
change. */
@@ -472,7 +612,7 @@ segmentation_on_threads(void *in_prm)
/* Grow the true clumps over the detection. */
clumps_grow_prepare_initial(&cltprm);
if(cltprm.diffuseindexs->size)
- clumps_grow(p->olabel, cltprm.diffuseindexs, 1, 1);
+ gal_label_grow_indexs(p->olabel, cltprm.diffuseindexs, 1, 1);
if(clprm->step==3)
{ gal_data_free(cltprm.diffuseindexs); continue; }
@@ -488,7 +628,7 @@ segmentation_on_threads(void *in_prm)
/* Identify the objects in this detection using the grown clumps
and correct the grown clump labels into new object labels. */
- segmentation_relab_to_objects(&cltprm);
+ segment_relab_to_objects(&cltprm);
if(clprm->step==4)
{
gal_data_free(cltprm.clumptoobj);
@@ -500,7 +640,7 @@ segmentation_on_threads(void *in_prm)
the diffuse indexs any more, so after filling the detected
region, free the indexs. */
if( cltprm.numobjects == 1 )
- segmentation_relab_noseg(&cltprm);
+ segment_relab_noseg(&cltprm);
else
{
/* Correct the labels so every non-labeled pixel can be
@@ -509,8 +649,8 @@ segmentation_on_threads(void *in_prm)
/* Cover the whole area (using maximum connectivity to not
miss any pixels). */
- clumps_grow(p->olabel, cltprm.diffuseindexs, 0,
- p->olabel->ndim);
+ gal_label_grow_indexs(p->olabel, cltprm.diffuseindexs, 0,
+ p->olabel->ndim);
/* Make sure all diffuse pixels are labeled. */
if(cltprm.diffuseindexs->size)
@@ -526,13 +666,13 @@ segmentation_on_threads(void *in_prm)
when there is more than object over the detection or when
there were multiple clumps over the detection. */
if(cltprm.numobjects>1)
- segmentation_relab_clumps_in_objects(&cltprm);
+ segment_relab_clumps_in_objects(&cltprm);
gal_data_free(cltprm.clumptoobj);
if(clprm->step==6) {continue;}
}
/* Convert the object labels to their final value */
- segmentation_relab_overall(&cltprm);
+ segment_relab_overall(&cltprm);
}
/* Wait until all the threads finish then return. */
@@ -547,7 +687,7 @@ segmentation_on_threads(void *in_prm)
/* If the user wanted to see the S/N table in a file, this function will be
called and will do the job. */
static void
-segmentation_save_sn_table(struct clumps_params *clprm)
+segment_save_sn_table(struct clumps_params *clprm)
{
char *msg;
float *sarr;
@@ -555,7 +695,7 @@ segmentation_save_sn_table(struct clumps_params *clprm)
gal_list_str_t *comments=NULL;
size_t i, j, c=0, totclumps=0;
gal_data_t *sn, *objind, *clumpinobj;
- struct noisechiselparams *p=clprm->p;
+ struct segmentparams *p=clprm->p;
/* Find the total number of clumps in all the initial detections. Recall
@@ -594,7 +734,7 @@ segmentation_save_sn_table(struct clumps_params *clprm)
gal_list_str_add(&comments, "See also: `CLUMPS_ALL_DET' HDU of "
"output with `--checksegmentation'.", 1);
if( asprintf(&msg, "S/N values of `nan': clumps smaller than "
- "`--segsnminarea' of %zu.", p->segsnminarea)<0 )
+ "`--snminarea' of %zu.", p->snminarea)<0 )
error(EXIT_FAILURE, 0, "%s: asprintf allocation", __func__);
gal_list_str_add(&comments, msg, 0);
gal_list_str_add(&comments, "S/N of clumps over detected regions.", 1);
@@ -628,7 +768,7 @@ segmentation_save_sn_table(struct clumps_params *clprm)
/* Find true clumps over the detected regions. */
static void
-segmentation_detections(struct noisechiselparams *p)
+segment_detections(struct segmentparams *p)
{
char *msg;
struct clumps_params clprm;
@@ -677,7 +817,7 @@ segmentation_detections(struct noisechiselparams *p)
claborig->size*gal_type_sizeof(claborig->type));
/* (Re-)do everything until this step. */
- gal_threads_spin_off(segmentation_on_threads, &clprm,
+ gal_threads_spin_off(segment_on_threads, &clprm,
p->numdetections, p->cp.numthreads);
/* Set the extension name. */
@@ -716,7 +856,8 @@ segmentation_detections(struct noisechiselparams *p)
demo->name = "DET_CLUMPS_GROWN";
if(!p->cp.quiet)
{
- gal_timing_report(NULL, "Starting to identify objects.", 1);
+ gal_timing_report(NULL, "Identify objects...",
+ 1);
if( asprintf(&msg, "True clumps grown "
"(HDU: `%s').", demo->name)<0 )
error(EXIT_FAILURE, 0, "%s: asprintf allocation",
@@ -799,7 +940,7 @@ segmentation_detections(struct noisechiselparams *p)
out of the loop, we don't need the rest of the process any
more. */
if( clprm.step==1
- && ( p->checkclumpsn && !p->continueaftercheck ) ) break;
+ && ( p->checksn && !p->continueaftercheck ) ) break;
/* Increment the step counter. */
++clprm.step;
@@ -812,7 +953,7 @@ segmentation_detections(struct noisechiselparams *p)
else
{
clprm.step=0;
- gal_threads_spin_off(segmentation_on_threads, &clprm, p->numdetections,
+ gal_threads_spin_off(segment_on_threads, &clprm, p->numdetections,
p->cp.numthreads);
}
@@ -822,7 +963,7 @@ segmentation_detections(struct noisechiselparams *p)
p->numobjects=clprm.totobjects;
/* If the user wanted to see the S/N table, then make the S/N table. */
- if(p->checkclumpsn) segmentation_save_sn_table(&clprm);
+ if(p->checksn) segment_save_sn_table(&clprm);
/* Clean up allocated structures and destroy the mutex. */
@@ -851,27 +992,124 @@ segmentation_detections(struct noisechiselparams *p)
/***********************************************************************/
+/***************** Output *****************/
+/***********************************************************************/
+void
+segment_output(struct segmentparams *p)
+{
+ float *f, *ff;
+ gal_fits_list_key_t *keys=NULL;
+
+ /* The Sky-subtracted input (if requested). */
+ if(!p->rawoutput)
+ gal_fits_img_write(p->input, p->cp.output, NULL, PROGRAM_NAME);
+
+
+ /* The clump labels. */
+ gal_fits_key_list_add(&keys, GAL_TYPE_FLOAT32, "CLUMPSN", 0,
+ &p->clumpsnthresh, 0, "Minimum S/N of true clumps",
+ 0, "ratio");
+ gal_fits_key_list_add(&keys, GAL_TYPE_SIZE_T, "NUMLABS", 0,
+ &p->numclumps, 0, "Total number of clumps", 0,
+ "counter");
+ p->clabel->name="CLUMPS";
+ gal_fits_img_write(p->clabel, p->cp.output, keys, PROGRAM_NAME);
+ p->clabel->name=NULL;
+ keys=NULL;
+
+
+ /* The object labels. */
+ gal_fits_key_list_add(&keys, GAL_TYPE_SIZE_T, "NUMLABS", 0,
+ &p->numobjects, 0, "Total number of objects", 0,
+ "counter");
+ p->olabel->name="OBJECTS";
+ gal_fits_img_write(p->olabel, p->cp.output, keys, PROGRAM_NAME);
+ p->olabel->name=NULL;
+ keys=NULL;
+
+
+ /* The Standard deviation image (if one was actually given). */
+ if( !p->rawoutput && p->std)
+ {
+ /* See if any keywords should be written (possibly inherited from the
+ detection program). */
+ if( !isnan(p->maxstd) )
+ gal_fits_key_list_add(&keys, GAL_TYPE_FLOAT32, "MAXSTD", 0,
+ &p->maxstd, 0,
+ "Maximum raw tile standard deviation", 0,
+ p->input->unit);
+ if( !isnan(p->minstd) )
+ gal_fits_key_list_add(&keys, GAL_TYPE_FLOAT32, "MINSTD", 0,
+ &p->minstd, 0,
+ "Minimum raw tile standard deviation", 0,
+ p->input->unit);
+ if( !isnan(p->medstd) )
+ gal_fits_key_list_add(&keys, GAL_TYPE_FLOAT32, "MEDSTD", 0,
+ &p->medstd, 0,
+ "Median raw tile standard deviation", 0,
+ p->input->unit);
+
+ /* If the input was actually a variance dataset, we'll need to take
+ its square root before writing it. We want this output to be a
+ standard deviation dataset. */
+ if(p->variance)
+ { ff=(f=p->std->array)+p->std->size; do *f=sqrt(*f); while(++f<ff); }
+
+ /* Write the STD dataset into the output file. */
+ p->std->name="SKY_STD";
+ if(p->std->size == p->input->size)
+ gal_fits_img_write(p->std, p->cp.output, keys, PROGRAM_NAME);
+ else
+ gal_tile_full_values_write(p->std, &p->cp.tl, 1, p->cp.output, keys,
+ PROGRAM_NAME);
+ p->std->name=NULL;
+ }
+
+ /* Let the user know that the output is written. */
+ if(!p->cp.quiet)
+ printf(" - Output written to `%s'.\n", p->cp.output);
+}
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+/***********************************************************************/
/***************** High level function *****************/
/***********************************************************************/
void
-segmentation(struct noisechiselparams *p)
+segment(struct segmentparams *p)
{
float *f;
char *msg;
- int32_t *l, *lf;
+ int32_t *c, *cf;
struct timeval t1;
- /* To keep the user up to date. */
- if(!p->cp.quiet)
- {
- if(!p->cp.quiet) gettimeofday(&t1, NULL);
- gal_timing_report(NULL, "Starting segmentation.",
- 1);
- }
+ /* Get starting time for later reporting if necessary. */
+ if(!p->cp.quiet) gettimeofday(&t1, NULL);
+
+ /* Prepare the inputs. */
+ segment_convolve(p);
+ segment_initialize(p);
- /* If a check segmentation image was requested, then put in the
- inputs. */
+
+ /* If a check segmentation image was requested, then start filling it
+ in. */
if(p->segmentationname)
{
gal_fits_img_write(p->input, p->segmentationname, NULL, PROGRAM_NAME);
@@ -882,31 +1120,34 @@ segmentation(struct noisechiselparams *p)
PROGRAM_NAME);
p->olabel->name=NULL;
}
-
-
- /* Allocate the clump labels image. */
- p->clabel=gal_data_alloc(NULL, p->olabel->type, p->olabel->ndim,
- p->olabel->dsize, p->olabel->wcs, 1,
- p->cp.minmapsize, NULL, NULL, NULL);
- p->clabel->flag=p->input->flag;
-
-
- /* Set any possibly existing NaN values to blank. */
- f=p->input->array; lf=(l=p->clabel->array)+p->clabel->size;
- do if(isnan(*f++)) *l=GAL_BLANK_INT32; while(++l<lf);
+ if(!p->cp.quiet)
+ printf(" - Input number of connected components: %zu\n",
+ p->numdetections);
/* Find the clump S/N threshold. */
- clumps_true_find_sn_thresh(p);
+ if( isnan(p->clumpsnthresh) )
+ {
+ gal_timing_report(NULL, "Finding true clumps...", 1);
+ clumps_true_find_sn_thresh(p);
+ }
+ else
+ {
+ if( asprintf(&msg, "Given S/N for true clumps: %g",
+ p->clumpsnthresh) <0 )
+ error(EXIT_FAILURE, 0, "%s: asprintf allocation", __func__);
+ gal_timing_report(NULL, msg, 1);
+ free(msg);
+ }
/* Reset the clabel array to find true clumps in objects. */
- f=p->input->array; lf=(l=p->clabel->array)+p->clabel->size;
- do *l = isnan(*f++) ? GAL_BLANK_INT32 : 0; while(++l<lf);
+ f=p->input->array; cf=(c=p->clabel->array)+p->clabel->size;
+ do *c = isnan(*f++) ? GAL_BLANK_INT32 : 0; while(++c<cf);
/* Find true clumps over the detected regions. */
- segmentation_detections(p);
+ segment_detections(p);
/* Report the results and timing to the user. */
@@ -926,4 +1167,8 @@ segmentation(struct noisechiselparams *p)
if(p->segmentationname && !p->continueaftercheck)
ui_abort_after_check(p, p->segmentationname, NULL,
"showing all segmentation steps");
+
+
+ /* Write the output. */
+ segment_output(p);
}
diff --git a/bin/noisechisel/segmentation.h b/bin/segment/segment.h
similarity index 80%
rename from bin/noisechisel/segmentation.h
rename to bin/segment/segment.h
index 6e17401..2e73e06 100644
--- a/bin/noisechisel/segmentation.h
+++ b/bin/segment/segment.h
@@ -1,6 +1,6 @@
/*********************************************************************
-NoiseChisel - Detect and segment signal in a noisy dataset.
-NoiseChisel is part of GNU Astronomy Utilities (Gnuastro) package.
+Segment - Segment initial labels based on signal structure.
+Segment is part of GNU Astronomy Utilities (Gnuastro) package.
Original author:
Mohammad Akhlaghi <mohammad@akhlaghi.org>
@@ -20,10 +20,10 @@ General Public License for more details.
You should have received a copy of the GNU General Public License
along with Gnuastro. If not, see <http://www.gnu.org/licenses/>.
**********************************************************************/
-#ifndef SEGMENTATION_H
-#define SEGMENTATION_H
+#ifndef SEGMENT_H
+#define SEGMENT_H
void
-segmentation(struct noisechiselparams *p);
+segment(struct segmentparams *p);
#endif
diff --git a/bin/segment/ui.c b/bin/segment/ui.c
new file mode 100644
index 0000000..5982fc9
--- /dev/null
+++ b/bin/segment/ui.c
@@ -0,0 +1,1051 @@
+/*********************************************************************
+Segment - Segment initial labels based on signal structure.
+Segment is part of GNU Astronomy Utilities (Gnuastro) package.
+
+Original author:
+ Mohammad Akhlaghi <mohammad@akhlaghi.org>
+Contributing author(s):
+Copyright (C) 2018, Free Software Foundation, Inc.
+
+Gnuastro is free software: you can redistribute it and/or modify it
+under the terms of the GNU General Public License as published by the
+Free Software Foundation, either version 3 of the License, or (at your
+option) any later version.
+
+Gnuastro is distributed in the hope that it will be useful, but
+WITHOUT ANY WARRANTY; without even the implied warranty of
+MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the GNU
+General Public License for more details.
+
+You should have received a copy of the GNU General Public License
+along with Gnuastro. If not, see <http://www.gnu.org/licenses/>.
+**********************************************************************/
+#include <config.h>
+
+#include <argp.h>
+#include <errno.h>
+#include <error.h>
+#include <stdio.h>
+#include <string.h>
+
+#include <gnuastro/wcs.h>
+#include <gnuastro/fits.h>
+#include <gnuastro/array.h>
+#include <gnuastro/binary.h>
+#include <gnuastro/threads.h>
+#include <gnuastro/dimension.h>
+#include <gnuastro/statistics.h>
+
+#include <gnuastro-internal/timing.h>
+#include <gnuastro-internal/options.h>
+#include <gnuastro-internal/checkset.h>
+#include <gnuastro-internal/fixedstringmacros.h>
+
+#include "main.h"
+
+#include "ui.h"
+#include "authors-cite.h"
+
+
+
+
+
+/**************************************************************/
+/********* Argp necessary global entities ************/
+/**************************************************************/
+/* Definition parameters for the Argp: */
+const char *
+argp_program_version = PROGRAM_STRING "\n"
+ GAL_STRINGS_COPYRIGHT
+ "\n\nWritten/developed by "PROGRAM_AUTHORS;
+
+const char *
+argp_program_bug_address = PACKAGE_BUGREPORT;
+
+static char
+args_doc[] = "ASTRdata";
+
+const char
+doc[] = GAL_STRINGS_TOP_HELP_INFO PROGRAM_NAME" will segment an initially "
+ "labeled region based on structure with the signal. It will first find "
+ "true clumps (local maxima), estimate which ones have strong connections, "
+ "and then grow them to cover the full area of each detection.\n"
+ GAL_STRINGS_MORE_HELP_INFO
+ /* After the list of options: */
+ "\v"
+ PACKAGE_NAME" home page: "PACKAGE_URL;
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+/**************************************************************/
+/********* Initialize & Parse command-line **************/
+/**************************************************************/
+static void
+ui_initialize_options(struct segmentparams *p,
+ struct argp_option *program_options,
+ struct argp_option *gal_commonopts_options)
+{
+ size_t i;
+ struct gal_options_common_params *cp=&p->cp;
+
+
+ /* Set the necessary common parameters structure. */
+ cp->poptions = program_options;
+ cp->program_name = PROGRAM_NAME;
+ cp->program_exec = PROGRAM_EXEC;
+ cp->program_bibtex = PROGRAM_BIBTEX;
+ cp->program_authors = PROGRAM_AUTHORS;
+ cp->numthreads = gal_threads_number();
+ cp->coptions = gal_commonopts_options;
+
+ p->skyval = NAN;
+ p->stdval = NAN;
+ p->medstd = NAN;
+ p->minstd = NAN;
+ p->maxstd = NAN;
+ p->clumpsnthresh = NAN;
+
+ /* Modify common options. */
+ for(i=0; !gal_options_is_last(&cp->coptions[i]); ++i)
+ {
+ /* Select individually. */
+ switch(cp->coptions[i].key)
+ {
+ case GAL_OPTIONS_KEY_LOG:
+ case GAL_OPTIONS_KEY_TYPE:
+ case GAL_OPTIONS_KEY_SEARCHIN:
+ case GAL_OPTIONS_KEY_IGNORECASE:
+ cp->coptions[i].flags=OPTION_HIDDEN;
+ break;
+
+ case GAL_OPTIONS_KEY_TILESIZE:
+ case GAL_OPTIONS_KEY_MINMAPSIZE:
+ case GAL_OPTIONS_KEY_NUMCHANNELS:
+ case GAL_OPTIONS_KEY_INTERPNUMNGB:
+ case GAL_OPTIONS_KEY_REMAINDERFRAC:
+ cp->coptions[i].mandatory=GAL_OPTIONS_MANDATORY;
+ break;
+
+ case GAL_OPTIONS_KEY_TABLEFORMAT:
+ cp->coptions[i].mandatory=GAL_OPTIONS_MANDATORY;
+ cp->coptions[i].doc="`txt', `fits-ascii', `fits-binary'.";
+ break;
+ }
+ }
+}
+
+
+
+
+
+/* Parse a single option: */
+error_t
+parse_opt(int key, char *arg, struct argp_state *state)
+{
+ struct segmentparams *p = state->input;
+
+ /* Pass `gal_options_common_params' into the child parser. */
+ state->child_inputs[0] = &p->cp;
+
+ /* In case the user incorrectly uses the equal sign (for example
+ with a short format or with space in the long format, then `arg`
+ start with (if the short version was called) or be (if the long
+ version was called with a space) the equal sign. So, here we
+ check if the first character of arg is the equal sign, then the
+ user is warned and the program is stopped: */
+ if(arg && arg[0]=='=')
+ argp_error(state, "incorrect use of the equal sign (`=`). For short "
+ "options, `=` should not be used and for long options, "
+ "there should be no space between the option, equal sign "
+ "and value");
+
+ /* Set the key to this option. */
+ switch(key)
+ {
+
+ /* Read the non-option tokens (arguments): */
+ case ARGP_KEY_ARG:
+ if(p->inputname)
+ argp_error(state, "only one argument (input file) should be given");
+ else
+ p->inputname=arg;
+ break;
+
+
+ /* This is an option, set its value. */
+ default:
+ return gal_options_set_from_key(key, arg, p->cp.poptions, &p->cp);
+ }
+
+ return 0;
+}
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+/**************************************************************/
+/*************** Sanity Check *******************/
+/**************************************************************/
+/* Read and check ONLY the options. When arguments are involved, do the
+ check in `ui_check_options_and_arguments'. */
+static void
+ui_read_check_only_options(struct segmentparams *p)
+{
+ char *tailptr;
+
+ /* If the full area is to be used as a single detection, we can't find
+ the S/N value from the un-detected regions, so the user must have
+ given the `clumpsnthresh' option. */
+ if( p->detectionname
+ && !strcmp(p->detectionname, DETECTION_ALL)
+ && isnan(p->clumpsnthresh) )
+ error(EXIT_FAILURE, 0, "`--clumpsnthresh' (`-%c') not given.\n\n"
+ "When `--detection=all' (the whole input dataset is assumed to "
+ "be a detection), Segment can't use the undetected pixels to find "
+ "the signal-to-noise ratio of true clumps. Therefore it is "
+ "mandatory to provide a signal-to-noise ratio manually",
+ UI_KEY_CLUMPSNTHRESH);
+
+ /* See if `--sky' is a filename or a value. When the string is only a
+ number (and nothing else), `tailptr' will point to the end of the
+ string (`\0'). When the string doesn't start with a number, it will
+ point to the start of the string. However, file names might also be
+ things like `1_sky.fits'. In such cases, `strtod' will return `1.0'
+ and `tailptr' will be `_std.fits', so we need to reset `p->stdval' to
+ NaN. */
+ if(p->skyname)
+ {
+ p->skyval=strtod(p->skyname, &tailptr);
+ if(tailptr==p->skyname || *tailptr!='\0')
+ p->skyval=NAN;
+ }
+
+ /* Similar to `--sky' above. */
+ if(p->stdname)
+ {
+ p->stdval=strtod(p->stdname, &tailptr);
+ if(tailptr==p->stdname || *tailptr!='\0')
+ p->stdval=NAN;
+ }
+
+ /* If the convolved HDU is given. */
+ if(p->convolvedname && p->chdu==NULL)
+ error(EXIT_FAILURE, 0, "no value given to `--convolvedhdu'. When the "
+ "`--convolved' option is called (to specify a convolved dataset "
+ "and avoid convolution) it is mandatory to also specify a HDU "
+ "for it");
+
+ /* For the options that make tables, the table format option is
+ mandatory. */
+ if( p->checksn && p->cp.tableformat==0 )
+ error(EXIT_FAILURE, 0, "`--tableformat' is necessary with the "
+ "`--checksn' option.\n"
+ "Please see description for `--tableformat' after running the "
+ "following command for more information (use `SPACE' to go down "
+ "the page and `q' to return to the command-line):\n\n"
+ " $ info gnuastro \"Input Output options\"");
+
+ /* Kernel checks. */
+ if(p->kernelname && strcmp(p->kernelname, UI_NO_CONV_KERNEL_NAME))
+ {
+ /* Check if it exists. */
+ gal_checkset_check_file(p->kernelname);
+
+ /* If its FITS, see if a HDU has been provided. */
+ if( gal_fits_name_is_fits(p->kernelname) && p->khdu==NULL )
+ error(EXIT_FAILURE, 0, "no HDU specified for kernel. When the "
+ "kernel is a FITS file, a HDU must also be specified. You "
+ "can use the `--khdu' option and give it the HDU number "
+ "(starting from zero), extension name, or anything "
+ "acceptable by CFITSIO");
+ }
+}
+
+
+
+
+
+static void
+ui_check_options_and_arguments(struct segmentparams *p)
+{
+ /* Make sure an input file name was given and if it was a FITS file, that
+ a HDU is also given. */
+ if(p->inputname)
+ {
+ /* Check if it exists. */
+ gal_checkset_check_file(p->inputname);
+
+ /* If it is FITS, a HDU is also mandatory. */
+ if( gal_fits_name_is_fits(p->inputname) && p->cp.hdu==NULL )
+ error(EXIT_FAILURE, 0, "no HDU specified. When the input is a FITS "
+ "file, a HDU must also be specified, you can use the `--hdu' "
+ "(`-h') option and give it the HDU number (starting from "
+ "zero), extension name, or anything acceptable by CFITSIO");
+
+ }
+ else
+ error(EXIT_FAILURE, 0, "no input file is specified");
+}
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+/**************************************************************/
+/*************** Preparations *******************/
+/**************************************************************/
+static void
+ui_set_used_names(struct segmentparams *p)
+{
+ p->useddetectionname = p->detectionname ? p->detectionname : p->inputname;
+
+ p->usedstdname = ( p->stdname
+ ? p->stdname
+ : ( ( p->detectionname
+ && strcmp(p->detectionname, DETECTION_ALL) )
+ ? p->detectionname
+ : p->inputname ) );
+}
+
+
+
+
+
+static void
+ui_set_output_names(struct segmentparams *p)
+{
+ char *output=p->cp.output;
+ char *basename = output ? output : p->inputname;
+
+ /* Main program output. */
+ if(output)
+ {
+ /* Delete the file if it already exists. */
+ gal_checkset_writable_remove(p->cp.output, 0, p->cp.dontdelete);
+
+ /* When the output name is given (possibly with directory
+ information), the check images will also be put in that same
+ directory. */
+ p->cp.keepinputdir=1;
+ }
+ else
+ p->cp.output=gal_checkset_automatic_output(&p->cp, p->inputname,
+ "_segmented.fits");
+
+ /* Tile check. */
+ if(p->cp.tl.checktiles)
+ p->cp.tl.tilecheckname=gal_checkset_automatic_output(&p->cp, basename,
+ "_tiles.fits");
+
+ /* Clump S/N values. */
+ if(p->checksn)
+ {
+ p->clumpsn_s_name=gal_checkset_automatic_output(&p->cp, basename,
+ ( p->cp.tableformat==GAL_TABLE_FORMAT_TXT
+ ? "_clumpsn_sky.txt" : "_clumpsn_sky.fits") );
+ p->clumpsn_d_name=gal_checkset_automatic_output(&p->cp, basename,
+ ( p->cp.tableformat==GAL_TABLE_FORMAT_TXT
+ ? "_clumpsn_det.txt" : "_clumpsn_det.fits") );
+ }
+
+ /* Segmentation steps. */
+ if(p->checksegmentation)
+ p->segmentationname=gal_checkset_automatic_output(&p->cp, basename,
+ "_segcheck.fits");
+}
+
+
+
+
+
+static void
+ui_prepare_inputs(struct segmentparams *p)
+{
+ int32_t *i, *ii;
+ gal_data_t *maxd, *ccin, *ccout=NULL;
+
+ /* Read the input as a single precision floating point dataset. */
+ p->input = gal_array_read_one_ch_to_type(p->inputname, p->cp.hdu,
+ GAL_TYPE_FLOAT32,
+ p->cp.minmapsize);
+ p->input->wcs = gal_wcs_read(p->inputname, p->cp.hdu, 0, 0,
+ &p->input->nwcs);
+ if(p->input->name) free(p->input->name);
+ gal_checkset_allocate_copy("INPUT", &p->input->name);
+
+
+ /* Check for blank values to help later processing. */
+ gal_blank_present(p->input, 1);
+
+
+ /* Segment currently only works on 2D datasets (images). */
+ if(p->input->ndim!=2 && p->input->ndim!=3)
+ error(EXIT_FAILURE, 0, "%s (hdu: %s) has %zu dimensions but Segment "
+ "can only operate on 2D (images) or 3D (cube) datasets",
+ p->inputname, p->cp.hdu, p->input->ndim);
+
+
+ /* If a convolved image is given, read it. */
+ if(p->convolvedname)
+ {
+ /* Read the input convolved image. */
+ p->conv = gal_array_read_one_ch_to_type(p->convolvedname, p->chdu,
+ GAL_TYPE_FLOAT32,
+ p->cp.minmapsize);
+ p->conv->wcs=gal_wcs_copy(p->input->wcs);
+
+ /* Make sure it is the same size as the input. */
+ if( gal_data_dsize_is_different(p->input, p->conv) )
+ error(EXIT_FAILURE, 0, "%s (hdu %s), given to `--convolved' and "
+ "`--chdu', is not the same size as the input (%s, hdu: %s)",
+ p->convolvedname, p->chdu, p->inputname, p->cp.hdu);
+ }
+
+
+ /* Read the detected label image and check its size. When the user gives
+ `--detection=all', then the whole input is assumed to be a single
+ detection. */
+ if( strcmp(p->useddetectionname, DETECTION_ALL) )
+ {
+ /* Read the dataset into memory. */
+ p->olabel = gal_array_read_one_ch(p->useddetectionname, p->dhdu,
+ p->cp.minmapsize);
+ if( gal_data_dsize_is_different(p->input, p->olabel) )
+ error(EXIT_FAILURE, 0, "`%s' (hdu: %s) and `%s' (hdu: %s) have a"
+ "different dimension/size", p->useddetectionname, p->dhdu,
+ p->inputname, p->cp.hdu);
+
+ /* Make sure the detected labels are not floating point. */
+ if(p->olabel->type==GAL_TYPE_FLOAT32
+ || p->olabel->type==GAL_TYPE_FLOAT64)
+ error(EXIT_FAILURE, 0, "%s (hdu: %s) has a `%s' type. The detection "
+ "(labeled) map must have an integer type (labels/classes can "
+ "only be integers). If the pixel values are integers, but only "
+ "the numerical type of the image is floating-point, you can "
+ "use the command below to convert it to a 32-bit (signed) "
+ "integer type:\n\n"
+ " $ astarithmetic %s int32 -h%s\n\n", p->useddetectionname,
+ p->dhdu, gal_type_name(p->olabel->type, 1),
+ p->useddetectionname, p->dhdu);
+
+ /* Get the maximum value of the input (total number of labels if they
+ are separate). If the maximum is 1 (the image is a binary image),
+ then apply the connected components algorithm to separate the
+ connected regions. The user is allowed to supply a simple binary
+ image.*/
+ maxd=gal_statistics_maximum(p->olabel);
+ maxd=gal_data_copy_to_new_type_free(maxd, GAL_TYPE_INT64);
+ p->numdetections = *((uint64_t *)(maxd->array));
+ if( p->numdetections == 1 )
+ {
+ ccin=gal_data_copy_to_new_type_free(p->olabel, GAL_TYPE_UINT8);
+ p->numdetections=gal_binary_connected_components(ccin, &ccout,
+ ccin->ndim);
+ gal_data_free(ccin);
+ p->olabel=ccout;
+ }
+ else
+ p->olabel = gal_data_copy_to_new_type_free(p->olabel, GAL_TYPE_INT32);
+
+
+ /* Write the WCS into the objects dataset too. */
+ p->olabel->wcs=gal_wcs_copy(p->input->wcs);
+ }
+ else
+ {
+ /* Set the total number of detections to 1. */
+ p->numdetections=1;
+
+ /* Allocate the array. */
+ p->olabel=gal_data_alloc(NULL, GAL_TYPE_INT32, p->input->ndim,
+ p->input->dsize, p->input->wcs, 0,
+ p->cp.minmapsize, NULL, NULL, NULL);
+
+ /* Initialize it to 1. */
+ ii=(i=p->olabel->array)+p->olabel->size; do *i++=1; while(i<ii);
+ }
+}
+
+
+
+
+
+/* Prepare the kernel, either from a file, or from the default arrays
+ available in the headers. The default kernels were created as
+ follows. */
+static void
+ui_prepare_kernel(struct segmentparams *p)
+{
+ float *f, *ff, *k;
+ size_t ndim=p->input->ndim;
+
+/* Since the default kernel has to be identical between NoiseChisel and
+ Segment, we have defined it in a shared header file to be accessible by
+ both programs. */
+#include <gnuastro-internal/kernel-2d.h>
+#include <gnuastro-internal/kernel-3d.h>
+
+ /* If a kernel file is given, then use it. Otherwise, use the default
+ kernel. */
+ if(p->kernelname)
+ {
+ /* Read the kernel. */
+ if( strcmp(p->kernelname, UI_NO_CONV_KERNEL_NAME) )
+ p->kernel=gal_fits_img_read_kernel(p->kernelname, p->khdu,
+ p->cp.minmapsize);
+ else
+ p->kernel=NULL;
+
+ /* Make sure it has the same dimensions as the input. */
+ if( p->kernel->ndim != p->input->ndim )
+ error(EXIT_FAILURE, 0, "%s (hdu %s): is %zuD, however, %s (%s) is a"
+ "%zuD dataset", p->kernelname, p->khdu, p->kernel->ndim,
+ p->inputname, p->cp.hdu, p->input->ndim);
+ }
+ else
+ {
+ /* Allocate space for the kernel (we don't want to use the statically
+ allocated array. */
+ p->kernel=gal_data_alloc(NULL, GAL_TYPE_FLOAT32, p->input->ndim,
+ ndim==2 ? kernel_2d_dsize : kernel_3d_dsize,
+ NULL, 0, p->cp.minmapsize, NULL, NULL, NULL);
+
+ /* Copy the staticly allocated default array into `p->kernel'. */
+ k = ndim==2 ? kernel_2d : kernel_3d;
+ ff = (f=p->kernel->array) + p->kernel->size;
+ do *f=*k++; while(++f<ff);
+ }
+}
+
+
+
+
+
+/* Set up the tessellation. */
+static void
+ui_prepare_tiles(struct segmentparams *p)
+{
+ gal_data_t *check;
+ struct gal_tile_two_layer_params *tl=&p->cp.tl, *ltl=&p->ltl;
+
+
+ /* Check the tile parameters for the small tile sizes and make the tile
+ structure. */
+ gal_tile_full_sanity_check(p->inputname, p->cp.hdu, p->input, tl);
+ gal_tile_full_two_layers(p->input, tl);
+ gal_tile_full_permutation(tl);
+
+
+ /* Make the large tessellation, except for the size, the rest of the
+ parameters are the same as the small tile sizes. */
+ ltl->numchannels = tl->numchannels;
+ ltl->remainderfrac = tl->remainderfrac;
+ ltl->workoverch = tl->workoverch;
+ ltl->checktiles = tl->checktiles;
+ ltl->oneelempertile = tl->oneelempertile;
+ gal_tile_full_sanity_check(p->inputname, p->cp.hdu, p->input, ltl);
+ gal_tile_full_two_layers(p->input, ltl);
+ gal_tile_full_permutation(ltl);
+
+
+ /* If the input has blank elements, then set the appropriate flag for
+ each tile.*/
+ if( p->input->flag & GAL_DATA_FLAG_HASBLANK )
+ {
+ gal_tile_block_blank_flag(tl->tiles, p->cp.numthreads);
+ gal_tile_block_blank_flag(ltl->tiles, p->cp.numthreads);
+ }
+
+
+ /* Make the tile check image if requested. */
+ if(tl->checktiles)
+ {
+ /* Large tiles. */
+ check=gal_tile_block_check_tiles(ltl->tiles);
+ gal_fits_img_write(check, tl->tilecheckname, NULL, PROGRAM_NAME);
+ gal_data_free(check);
+
+ /* Small tiles. */
+ check=gal_tile_block_check_tiles(tl->tiles);
+ gal_fits_img_write(check, tl->tilecheckname, NULL, PROGRAM_NAME);
+ gal_data_free(check);
+
+ /* If `continueaftercheck' hasn't been called, abort NoiseChisel. */
+ if(!p->continueaftercheck)
+ ui_abort_after_check(p, tl->tilecheckname, NULL,
+ "showing all tiles over the image");
+
+ /* Free the name. */
+ free(tl->tilecheckname);
+ tl->tilecheckname=NULL;
+ }
+}
+
+
+
+
+
+static void
+ui_check_size(gal_data_t *base, gal_data_t *comp, size_t numtiles,
+ char *bname, char *bhdu, char *cname, char *chdu)
+{
+ if( gal_data_dsize_is_different(base, comp) && numtiles!=comp->size)
+ error(EXIT_FAILURE, 0, "%s (hdu: %s): doesn't have the right size "
+ "(%zu elements or pixels).\n\n"
+ "It must either be the same size as `%s' (hdu: `%s'), or "
+ "it must have the same number of elements as the total "
+ "number of tiles in the tessellation (%zu). In the latter "
+ "case, each pixel is assumed to be a fixed value for a "
+ "complete tile.\n\n"
+ "Run with `-P' to see the (tessellation) options/settings "
+ "and their values). For more information on tessellation in "
+ "Gnuastro, please run the following command (use the arrow "
+ "keys for up and down and press `q' to return to the "
+ "command-line):\n\n"
+ " $ info gnuastro tessellation",
+ cname, chdu, comp->size, bname, bhdu, numtiles);
+}
+
+
+
+
+
+/* Subtract `sky' from the input dataset depending on its size (it may be
+ the whole array or a tile-values array).. */
+static void
+ui_subtract_sky(gal_data_t *in, gal_data_t *sky,
+ struct gal_tile_two_layer_params *tl)
+{
+ size_t tid;
+ gal_data_t *tile;
+ float *s, *f, *ff, *skyarr=sky->array;
+
+ /* It is the same size as the input or a single value. */
+ if( gal_data_dsize_is_different(in, sky)==0 || sky->size==1)
+ {
+ s=sky->array;
+ ff=(f=in->array)+in->size;
+ if(sky->size==1) { if(*s!=0.0) do *f-=*s; while(++f<ff); }
+ else do *f-=*s++; while(++f<ff);
+ }
+
+ /* It is the same size as the number of tiles. */
+ else if( tl->tottiles==sky->size )
+ {
+ /* Go over all the tiles. */
+ for(tid=0; tid<tl->tottiles; ++tid)
+ {
+ /* For easy reading. */
+ tile=&tl->tiles[tid];
+
+ /* Subtract the Sky value from the input image. */
+ GAL_TILE_PARSE_OPERATE(tile, NULL, 0, 0, {*i-=skyarr[tid];});
+ }
+ }
+
+ /* The size must have been checked before, so if control reaches here, we
+ have a bug! */
+ else
+ error(EXIT_FAILURE, 0, "%s: a bug! Please contact us at %s to fix "
+ "the problem. For some reason, the size doesn't match", __func__,
+ PACKAGE_BUGREPORT);
+}
+
+
+
+
+
+/* The Sky and Sky standard deviation images can be a `oneelempertile'
+ image (only one element/pixel for a tile). So we need to do some extra
+ checks on them (after reading the tessellation). */
+static void
+ui_read_std_and_sky(struct segmentparams *p)
+{
+ size_t one=1;
+ struct gal_tile_two_layer_params *tl=&p->cp.tl;
+ gal_data_t *sky, *keys=gal_data_array_calloc(3);
+
+ /* Read the standard devitaion image (if it isn't a single number). */
+ if(isnan(p->stdval))
+ {
+ /* Make sure a HDU is also given. */
+ if(p->stdhdu==NULL)
+ error(EXIT_FAILURE, 0, "no value given to `--stdhdu'.\n\n"
+ "When the Sky standard deviation is a dataset, it is mandatory "
+ "specify which HDU/extension it is present in. The file can "
+ "be specified explicitly with `--std'. If not, segment will "
+ "use the file given to `--detection'. If that is also not "
+ "called, it will look into the main input file (with no "
+ "option)");
+
+ /* Read the STD image. */
+ p->std=gal_array_read_one_ch_to_type(p->usedstdname, p->stdhdu,
+ GAL_TYPE_FLOAT32,
+ p->cp.minmapsize);
+
+ /* Make sure it has the correct size. */
+ ui_check_size(p->input, p->std, tl->tottiles, p->inputname, p->cp.hdu,
+ p->usedstdname, p->stdhdu);
+ }
+
+
+ /* If a Sky image is given, subtract it from the values and convolved
+ images immediately and free it. */
+ if(p->skyname)
+ {
+ /* If its a file, read it into memory. */
+ if( isnan(p->skyval) )
+ {
+ /* Make sure a HDU is also given. */
+ if(p->skyhdu==NULL)
+ error(EXIT_FAILURE, 0, "no value given to `--skyhdu'.\n\n"
+ "When the Sky is a dataset, it is mandatory specify "
+ "which HDU/extension it is present in. The file can be "
+ "specified explicitly with `--sky'. If it is a single "
+ "value, you can just pass the value to `--sky' and no "
+ "HDU will be necessary");
+
+ /* Read the Sky dataset. */
+ sky=gal_array_read_one_ch_to_type(p->skyname, p->skyhdu,
+ GAL_TYPE_FLOAT32,
+ p->cp.minmapsize);
+
+ /* Check its size. */
+ ui_check_size(p->input, sky, tl->tottiles, p->inputname, p->cp.hdu,
+ p->skyname, p->skyhdu);
+ }
+ else
+ {
+ sky=gal_data_alloc(NULL, GAL_TYPE_FLOAT32, 1, &one, NULL, 0, -1,
+ NULL, NULL, NULL);
+ *((float *)(sky->array)) = p->skyval;
+ }
+
+ /* Subtract it from the input. */
+ ui_subtract_sky(p->input, sky, tl);
+
+ /* If a convolved image is given, subtract the Sky from that too. */
+ if(p->conv)
+ ui_subtract_sky(p->conv, sky, tl);
+
+ /* Clean up. */
+ gal_data_free(sky);
+ p->skyval=NAN;
+ }
+
+
+ /* When the Standard deviation image is made by NoiseChisel, it puts
+ three basic statistics of the pre-interpolation distribution of
+ standard deviations in `MEDSTD', `MINSTD' and `MAXSTD'. The `MEDSTD'
+ in particular is most important because it can't be inferred after the
+ interpolations and it can be useful in MakeCatalog later to give a
+ more accurate estimate of the noise level. So if they are present, we
+ will read them here and write them to the STD output (which is created
+ when `--rawoutput' is not given). */
+ if(!p->rawoutput && p->std)
+ {
+ keys[0].next=&keys[1];
+ keys[1].next=&keys[2];
+ keys[2].next=NULL;
+ keys[0].array=&p->medstd; keys[0].name="MEDSTD";
+ keys[1].array=&p->minstd; keys[1].name="MINSTD";
+ keys[2].array=&p->maxstd; keys[2].name="MAXSTD";
+ keys[0].type=keys[1].type=keys[2].type=GAL_TYPE_FLOAT32;
+ gal_fits_key_read(p->usedstdname, p->stdhdu, keys, 0, 0);
+ if(keys[0].status) p->medstd=NAN;
+ if(keys[1].status) p->minstd=NAN;
+ if(keys[2].status) p->maxstd=NAN;
+ keys[0].name=keys[1].name=keys[2].name=NULL;
+ keys[0].array=keys[1].array=keys[2].array=NULL;
+ gal_data_array_free(keys, 3, 1);
+ }
+}
+
+
+
+
+
+static void
+ui_preparations(struct segmentparams *p)
+{
+ /* Set the input names. */
+ ui_set_used_names(p);
+
+ /* Prepare the names of the outputs. */
+ ui_set_output_names(p);
+
+ /* Read the input datasets. */
+ ui_prepare_inputs(p);
+
+ /* If a convolved image was given, read it in. Otherwise, read the given
+ kernel. */
+ if(p->conv==NULL)
+ ui_prepare_kernel(p);
+
+ /* Prepare the tessellation. */
+ ui_prepare_tiles(p);
+
+ /* Prepare the (optional Sky, and) Sky Standard deviation image. */
+ ui_read_std_and_sky(p);
+}
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+/**************************************************************/
+/************ Set the parameters *************/
+/**************************************************************/
+void
+ui_read_check_inputs_setup(int argc, char *argv[], struct segmentparams *p)
+{
+ char *stdunit;
+ struct gal_options_common_params *cp=&p->cp;
+
+
+ /* Include the parameters necessary for argp from this program (`args.h')
+ and for the common options to all Gnuastro (`commonopts.h'). We want
+ to directly put the pointers to the fields in `p' and `cp', so we are
+ simply including the header here to not have to use long macros in
+ those headers which make them hard to read and modify. This also helps
+ in having a clean environment: everything in those headers is only
+ available within the scope of this function. */
+#include <gnuastro-internal/commonopts.h>
+#include "args.h"
+
+
+ /* Initialize the options and necessary information. */
+ ui_initialize_options(p, program_options, gal_commonopts_options);
+
+
+ /* Read the command-line options and arguments. */
+ errno=0;
+ if(argp_parse(&thisargp, argc, argv, 0, 0, p))
+ error(EXIT_FAILURE, errno, "parsing arguments");
+
+
+ /* Read the configuration files and set the common values. */
+ gal_options_read_config_set(&p->cp);
+
+
+ /* Read the options into the program's structure, and check them and
+ their relations prior to printing. */
+ ui_read_check_only_options(p);
+
+
+ /* Print the option values if asked. Note that this needs to be done
+ after the option checks so un-sane values are not printed in the
+ output state. */
+ gal_options_print_state(&p->cp);
+
+
+ /* Check that the options and arguments fit well with each other. Note
+ that arguments don't go in a configuration file. So this test should
+ be done after (possibly) printing the option values. */
+ ui_check_options_and_arguments(p);
+
+
+ /* Read/allocate all the necessary starting arrays. */
+ ui_preparations(p);
+
+
+ /* Let the user know that processing has started. */
+ if(!p->cp.quiet)
+ {
+ /* Basic inputs. */
+ printf(PROGRAM_NAME" started on %s", ctime(&p->rawtime));
+ printf(" - Using %zu CPU thread%s\n", p->cp.numthreads,
+ p->cp.numthreads==1 ? "." : "s.");
+ printf(" - Input: %s (hdu: %s)\n", p->inputname, p->cp.hdu);
+
+ /* Sky value information. */
+ if(p->skyname)
+ {
+ if( isnan(p->skyval) )
+ printf(" - Sky: %s (hdu: %s)\n", p->skyname, p->skyhdu);
+ else
+ printf(" - Sky: %g\n", p->skyval);
+ }
+
+ /* Sky Standard deviation information. */
+ stdunit = p->variance ? "variance" : "STD";
+ if(p->std)
+ printf(" - Sky %s: %s (hdu: %s)\n", stdunit, p->usedstdname,
+ p->stdhdu);
+ else
+ printf(" - Sky %s: %g\n", stdunit, p->stdval);
+
+ /* Convolution information. */
+ if(p->convolvedname)
+ printf(" - Convolved input: %s (hdu: %s)\n", p->convolvedname,
+ p->chdu);
+ else
+ {
+ if(p->kernelname)
+ {
+ if( strcmp(p->kernelname, UI_NO_CONV_KERNEL_NAME) )
+ printf(" - Kernel: %s (hdu: %s)\n", p->kernelname, p->khdu);
+ else
+ printf(" - No convolution requested.\n");
+ }
+ else
+ printf(" - Kernel: FWHM=2 pixel Gaussian.\n");
+ }
+ printf(" - Detection: %s (hdu: %s)\n", p->useddetectionname, p->dhdu);
+ }
+}
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+/**************************************************************/
+/************ Free allocated, report *************/
+/**************************************************************/
+void
+ui_abort_after_check(struct segmentparams *p, char *filename,
+ char *file2name, char *description)
+{
+ char *name;
+
+ if(file2name)
+ {
+ if( asprintf(&name, "`%s' and `%s'", filename, file2name)<0 )
+ error(EXIT_FAILURE, 0, "%s: asprintf allocation", __func__);
+ }
+ else
+ {
+ if( asprintf(&name, "`%s'", filename)<0 )
+ error(EXIT_FAILURE, 0, "%s: asprintf allocation", __func__);
+ }
+
+ /* Let the user know that NoiseChisel is aborting. */
+ fprintf(stderr,
+ "------------------------------------------------\n"
+ "%s aborted for a check\n"
+ "------------------------------------------------\n"
+ "%s (%s) has been created.\n\n"
+ "If you want %s to continue its processing AND save any "
+ "requested check outputs, please run it again with "
+ "`--continueaftercheck'.\n"
+ "------------------------------------------------\n",
+ PROGRAM_NAME, name, description, PROGRAM_NAME);
+
+ /* Clean up. */
+ free(name);
+ ui_free_report(p, NULL);
+
+ /* Abort. */
+ exit(EXIT_SUCCESS);
+}
+
+
+
+
+
+void
+ui_free_report(struct segmentparams *p, struct timeval *t1)
+{
+ /* Free the allocated arrays: */
+ free(p->cp.hdu);
+ free(p->cp.output);
+ gal_data_free(p->input);
+ gal_data_free(p->kernel);
+ gal_data_free(p->binary);
+ gal_data_free(p->olabel);
+ gal_data_free(p->clabel);
+ if(p->khdu) free(p->khdu);
+ if(p->chdu) free(p->chdu);
+ if(p->dhdu) free(p->dhdu);
+ if(p->skyhdu) free(p->skyhdu);
+ if(p->stdhdu) free(p->stdhdu);
+ if(p->stdname) free(p->stdname);
+ if(p->kernelname) free(p->kernelname);
+ if(p->detectionname) free(p->detectionname);
+ if(p->convolvedname) free(p->convolvedname);
+ if(p->conv!=p->input) gal_data_free(p->conv);
+ if(p->clumpsn_s_name) free(p->clumpsn_s_name);
+ if(p->clumpsn_d_name) free(p->clumpsn_d_name);
+ if(p->segmentationname) free(p->segmentationname);
+
+ /* Print the final message. */
+ if(!p->cp.quiet && t1)
+ gal_timing_report(t1, PROGRAM_NAME" finished in: ", 0);
+}
diff --git a/bin/noisechisel/ui.h b/bin/segment/ui.h
similarity index 54%
copy from bin/noisechisel/ui.h
copy to bin/segment/ui.h
index 6e33e94..79a2cc5 100644
--- a/bin/noisechisel/ui.h
+++ b/bin/segment/ui.h
@@ -1,11 +1,11 @@
/*********************************************************************
-NoiseChisel - Detect and segment signal in a noisy dataset.
-NoiseChisel is part of GNU Astronomy Utilities (Gnuastro) package.
+Segment - Segment initial labels based on signal structure.
+Segment is part of GNU Astronomy Utilities (Gnuastro) package.
Original author:
Mohammad Akhlaghi <mohammad@akhlaghi.org>
Contributing author(s):
-Copyright (C) 2015-2018, Free Software Foundation, Inc.
+Copyright (C) 2018, Free Software Foundation, Inc.
Gnuastro is free software: you can redistribute it and/or modify it
under the terms of the GNU General Public License as published by the
@@ -40,8 +40,7 @@ along with Gnuastro. If not, see
<http://www.gnu.org/licenses/>.
/* Option groups particular to this program. */
enum program_args_groups
{
- UI_GROUP_DETECTION = GAL_OPTIONS_GROUP_AFTER_COMMON,
- UI_GROUP_SEGMENTATION,
+ UI_GROUP_SEGMENTATION = GAL_OPTIONS_GROUP_AFTER_COMMON,
};
@@ -50,58 +49,40 @@ enum program_args_groups
/* Available letters for short options:
- a b f j l n u x z
- A H J W X Y
+ a b e f g i j l n p r t u w x z
+ A E H J Q R W X Y
*/
enum option_keys_enum
{
/* With short-option version. */
- UI_KEY_LARGETILESIZE = 'L',
UI_KEY_KERNEL = 'k',
- UI_KEY_WIDEKERNEL = 'w',
- UI_KEY_SKYSUBTRACTED = 'E',
+ UI_KEY_DETECTION = 'd',
+ UI_KEY_LARGETILESIZE = 'L',
UI_KEY_MINSKYFRAC = 'B',
- UI_KEY_MIRRORDIST = 'r',
- UI_KEY_MODMEDQDIFF = 'Q',
- UI_KEY_QTHRESH = 't',
- UI_KEY_ERODE = 'e',
- UI_KEY_OPENING = 'p',
- UI_KEY_SIGMACLIP = 's',
- UI_KEY_DTHRESH = 'R',
- UI_KEY_DETSNMINAREA = 'i',
- UI_KEY_DETQUANT = 'c',
- UI_KEY_DETGROWQUANT = 'd',
- UI_KEY_SEGSNMINAREA = 'm',
- UI_KEY_SEGQUANT = 'g',
+ UI_KEY_SNMINAREA = 'm',
+ UI_KEY_SNQUANT = 'c',
UI_KEY_KEEPMAXNEARRIVER = 'v',
+ UI_KEY_CLUMPSNTHRESH = 's',
UI_KEY_GTHRESH = 'G',
UI_KEY_MINRIVERLENGTH = 'y',
UI_KEY_OBJBORDERSN = 'O',
UI_KEY_CONTINUEAFTERCHECK = 'C',
-
/* Only with long version (start with a value 1000, the rest will be set
automatically). */
UI_KEY_KHDU = 1000,
UI_KEY_CONVOLVED,
- UI_KEY_CONVOLVEDHDU,
- UI_KEY_WKHDU,
+ UI_KEY_CHDU,
+ UI_KEY_DHDU,
+ UI_KEY_SKY,
+ UI_KEY_SKYHDU,
+ UI_KEY_STD,
+ UI_KEY_STDHDU,
+ UI_KEY_VARIANCE,
+ UI_KEY_RAWOUTPUT,
UI_KEY_MINNUMFALSE,
- UI_KEY_ONLYDETECTION,
UI_KEY_GROWNCLUMPS,
- UI_KEY_SMOOTHWIDTH,
- UI_KEY_QTHRESHTILEQUANT,
- UI_KEY_CHECKQTHRESH,
- UI_KEY_ERODENGB,
- UI_KEY_NOERODEQUANT,
- UI_KEY_OPENINGNGB,
- UI_KEY_CHECKDETSKY,
- UI_KEY_CHECKDETSN,
- UI_KEY_DETGROWMAXHOLESIZE,
- UI_KEY_CLEANGROWNDET,
- UI_KEY_CHECKDETECTION,
- UI_KEY_CHECKSKY,
- UI_KEY_CHECKCLUMPSN,
+ UI_KEY_CHECKSN,
UI_KEY_CHECKSEGMENTATION,
};
@@ -110,14 +91,13 @@ enum option_keys_enum
void
-ui_read_check_inputs_setup(int argc, char *argv[],
- struct noisechiselparams *p);
+ui_read_check_inputs_setup(int argc, char *argv[], struct segmentparams *p);
void
-ui_abort_after_check(struct noisechiselparams *p, char *filename,
- char *file2name, char *description);
+ui_abort_after_check(struct segmentparams *p, char *filename, char *file2name,
+ char *description);
void
-ui_free_report(struct noisechiselparams *p, struct timeval *t1);
+ui_free_report(struct segmentparams *p, struct timeval *t1);
#endif
diff --git a/bin/statistics/sky.c b/bin/statistics/sky.c
index a77d70a..ffd56c1 100644
--- a/bin/statistics/sky.c
+++ b/bin/statistics/sky.c
@@ -185,7 +185,7 @@ sky(struct statisticsparams *p)
{
num=gal_statistics_number(p->sky_t);
if( asprintf(&msg, "Sky and its STD found on %zu/%zu tiles.",
- (size_t)(*((uint64_t *)(num->array))), tl->tottiles )<0 )
+ *((size_t *)(num->array)), tl->tottiles )<0 )
error(EXIT_FAILURE, 0, "%s: asprintf allocation", __func__);
gal_timing_report(&t1, msg, 1);
gal_data_free(num);
diff --git a/bin/statistics/ui.c b/bin/statistics/ui.c
index bed2975..0784419 100644
--- a/bin/statistics/ui.c
+++ b/bin/statistics/ui.c
@@ -30,6 +30,7 @@ along with Gnuastro. If not, see
<http://www.gnu.org/licenses/>.
#include <gnuastro/wcs.h>
#include <gnuastro/fits.h>
#include <gnuastro/tile.h>
+#include <gnuastro/array.h>
#include <gnuastro/qsort.h>
#include <gnuastro/blank.h>
#include <gnuastro/table.h>
@@ -797,7 +798,9 @@ ui_preparations(struct statisticsparams *p)
if(p->isfits && p->hdu_type==IMAGE_HDU)
{
p->inputformat=INPUT_FORMAT_IMAGE;
- p->input=gal_fits_img_read(p->inputname, cp->hdu, cp->minmapsize, 0, 0);
+ p->input=gal_array_read_one_ch(p->inputname, cp->hdu, cp->minmapsize);
+ p->input->wcs=gal_wcs_read(p->inputname, cp->hdu, 0, 0,
+ &p->input->nwcs);
}
else
{
@@ -850,6 +853,12 @@ ui_preparations(struct statisticsparams *p)
{
/* Only keep the elements we want. */
gal_blank_remove(p->input);
+
+ /* Make sure there actually are any (non-blank) elements left. */
+ if(p->input->size==0)
+ error(EXIT_FAILURE, 0, "%s: all elements are blank",
+ gal_fits_name_save_as_string(p->inputname, cp->hdu));
+
p->input->flag &= ~GAL_DATA_FLAG_HASBLANK ;
p->input->flag |= GAL_DATA_FLAG_BLANK_CH ;
diff --git a/bin/table/ui.c b/bin/table/ui.c
index 58949e4..a401692 100644
--- a/bin/table/ui.c
+++ b/bin/table/ui.c
@@ -262,7 +262,7 @@ ui_check_options_and_arguments(struct tableparams *p)
/**************************************************************/
/*************** Preparations *******************/
/**************************************************************/
-void
+static void
ui_print_info_exit(struct tableparams *p)
{
char *tmp;
@@ -307,7 +307,51 @@ ui_print_info_exit(struct tableparams *p)
-void
+/* The columns can be given as comma-separated values to one option or
+ multiple calls to the column option. Here, we'll check if the input list
+ has comma-separated values. If they do then the list will be updated to
+ be fully separate. */
+static void
+ui_columns_prepare(struct tableparams *p)
+{
+ size_t i;
+ char **strarr;
+ gal_data_t *strs;
+ gal_list_str_t *tmp, *new=NULL;
+
+ /* Go over the whole original list (where each node may have more than
+ one value separated by a comma. */
+ for(tmp=p->columns;tmp!=NULL;tmp=tmp->next)
+ {
+ /* Read the different comma-separated strings into an array (within a
+ `gal_data_t'). */
+ strs=gal_options_parse_csv_strings_raw(tmp->v, NULL, 0);
+ strarr=strs->array;
+
+ /* Go over all the elements and add them to the `new' list. */
+ for(i=0;i<strs->size;++i)
+ {
+ gal_list_str_add(&new, strarr[i], 0);
+ strarr[i]=NULL;
+ }
+
+ /* Clean up. */
+ gal_data_free(strs);
+ }
+
+ /* Delete the old list. */
+ gal_list_str_free(p->columns, 1);
+
+ /* Reverse the new list, then put it into `p->columns'. */
+ gal_list_str_reverse(&new);
+ p->columns=new;
+}
+
+
+
+
+
+static void
ui_preparations(struct tableparams *p)
{
struct gal_options_common_params *cp=&p->cp;
@@ -317,6 +361,9 @@ ui_preparations(struct tableparams *p)
if(p->information)
ui_print_info_exit(p);
+ /* Prepare the column names. */
+ ui_columns_prepare(p);
+
/* Read in the table columns. */
p->table=gal_table_read(p->filename, cp->hdu, p->columns, cp->searchin,
cp->ignorecase, cp->minmapsize, NULL);
diff --git a/bin/warp/ui.c b/bin/warp/ui.c
index 742d6c7..632715a 100644
--- a/bin/warp/ui.c
+++ b/bin/warp/ui.c
@@ -30,6 +30,7 @@ along with Gnuastro. If not, see
<http://www.gnu.org/licenses/>.
#include <gnuastro/wcs.h>
#include <gnuastro/fits.h>
+#include <gnuastro/array.h>
#include <gnuastro/table.h>
#include <gnuastro/threads.h>
@@ -337,9 +338,11 @@ ui_check_options_and_arguments(struct warpparams *p)
"by CFITSIO)");
/* Read the input image as double type and its WCS structure. */
- p->input=gal_fits_img_read_to_type(p->inputname, p->cp.hdu,
- GAL_TYPE_FLOAT64, p->cp.minmapsize,
- p->hstartwcs, p->hendwcs);
+ p->input=gal_array_read_one_ch_to_type(p->inputname, p->cp.hdu,
+ GAL_TYPE_FLOAT64,
+ p->cp.minmapsize);
+ p->input->wcs=gal_wcs_read(p->inputname, p->cp.hdu, p->hstartwcs,
+ p->hendwcs, &p->input->nwcs);
if(p->input->wcs)
{
p->pixelscale=gal_wcs_pixel_scale(p->input->wcs);
@@ -890,6 +893,7 @@ ui_preparations(struct warpparams *p)
void
ui_read_check_inputs_setup(int argc, char *argv[], struct warpparams *p)
{
+ double *matrix;
struct gal_options_common_params *cp=&p->cp;
@@ -937,7 +941,7 @@ ui_read_check_inputs_setup(int argc, char *argv[], struct
warpparams *p)
/* Everything is ready, notify the user of the program starting. */
if(!p->cp.quiet)
{
- double *matrix=p->matrix->array;
+ matrix=p->matrix->array;
printf(PROGRAM_NAME" started on %s", ctime(&p->rawtime));
printf(" Using %zu CPU thread%s\n", p->cp.numthreads,
p->cp.numthreads==1 ? "." : "s.");
diff --git a/bin/warp/warp.c b/bin/warp/warp.c
index 77ddd6c..7065ec6 100644
--- a/bin/warp/warp.c
+++ b/bin/warp/warp.c
@@ -414,7 +414,7 @@ correct_wcs_save_output(struct warpparams *p)
double *m=p->matrix->array, diff;
struct wcsprm *wcs=p->output->wcs;
gal_fits_list_key_t *headers=NULL;
- double *crpix=wcs->crpix, *w=p->inwcsmatrix;
+ double *crpix=wcs?wcs->crpix:NULL, *w=p->inwcsmatrix;
/* `tinv' is the 2 by 2 inverse matrix. Recall that `p->inverse' is 3 by
3 to account for homogeneous coordinates. */
@@ -422,28 +422,41 @@ correct_wcs_save_output(struct warpparams *p)
p->inverse[3]/p->inverse[8], p->inverse[4]/p->inverse[8]};
/* Make the WCS corrections if necessary. */
- if(p->keepwcs==0 && wcs)
+ if(wcs)
{
- /* Correct the input WCS matrix. Since we are re-writing the PC
- matrix from the full rotation matrix (including pixel scale),
- we'll also have to set the CDELT fields to 1. Just to be sure that
- the PC matrix is used in the end by WCSLIB, we'll also set altlin
- to 1.*/
- wcs->altlin=1;
- wcs->cdelt[0] = wcs->cdelt[1] = 1.0f;
- wcs->pc[0] = w[0]*tinv[0] + w[1]*tinv[2];
- wcs->pc[1] = w[0]*tinv[1] + w[1]*tinv[3];
- wcs->pc[2] = w[2]*tinv[0] + w[3]*tinv[2];
- wcs->pc[3] = w[2]*tinv[1] + w[3]*tinv[3];
-
- /* Correct the CRPIX point. The +1 in the end of the last two
- lines is because FITS counts from 1. */
- tcrpix[0] = m[0]*crpix[0]+m[1]*crpix[1]+m[2];
- tcrpix[1] = m[3]*crpix[0]+m[4]*crpix[1]+m[5];
- tcrpix[2] = m[6]*crpix[0]+m[7]*crpix[1]+m[8];
-
- crpix[0] = tcrpix[0]/tcrpix[2] - p->outfpixval[0] + 1;
- crpix[1] = tcrpix[1]/tcrpix[2] - p->outfpixval[1] + 1;
+ if(p->keepwcs==0)
+ {
+ /* Correct the input WCS matrix. Since we are re-writing the PC
+ matrix from the full rotation matrix (including pixel scale),
+ we'll also have to set the CDELT fields to 1. Just to be sure
+ that the PC matrix is used in the end by WCSLIB, we'll also
+ set altlin to 1.*/
+ wcs->altlin=1;
+ wcs->cdelt[0] = wcs->cdelt[1] = 1.0f;
+ wcs->pc[0] = w[0]*tinv[0] + w[1]*tinv[2];
+ wcs->pc[1] = w[0]*tinv[1] + w[1]*tinv[3];
+ wcs->pc[2] = w[2]*tinv[0] + w[3]*tinv[2];
+ wcs->pc[3] = w[2]*tinv[1] + w[3]*tinv[3];
+
+ /* Correct the CRPIX point. The +1 in the end of the last two
+ lines is because FITS counts from 1. */
+ tcrpix[0] = m[0]*crpix[0]+m[1]*crpix[1]+m[2];
+ tcrpix[1] = m[3]*crpix[0]+m[4]*crpix[1]+m[5];
+ tcrpix[2] = m[6]*crpix[0]+m[7]*crpix[1]+m[8];
+
+ crpix[0] = tcrpix[0]/tcrpix[2] - p->outfpixval[0] + 1;
+ crpix[1] = tcrpix[1]/tcrpix[2] - p->outfpixval[1] + 1;
+ }
+
+ /* Due to floating point errors extremely small values of PC matrix
+ can be set to zero and extremely small differences between PC1_1
+ and PC2_2 can be ignored. The reason for all the `fabs' functions
+ is because the signs are usually different.*/
+ if( fabs(wcs->pc[1])<ABSOLUTEFLTERROR ) wcs->pc[1]=0.0f;
+ if( fabs(wcs->pc[2])<ABSOLUTEFLTERROR ) wcs->pc[2]=0.0f;
+ diff=fabs(wcs->pc[0])-fabs(wcs->pc[3]);
+ if( fabs(diff/p->pixelscale[0])<RELATIVEFLTERROR )
+ wcs->pc[3]=( (wcs->pc[3] < 0.0f ? -1.0f : 1.0f) * fabs(wcs->pc[0]) );
}
/* Add the appropriate headers: */
@@ -456,16 +469,6 @@ correct_wcs_save_output(struct warpparams *p)
"Warp matrix element value", 0, NULL);
}
- /* Due to floating point errors extremely small values of PC matrix can
- be set to zero and extremely small differences between PC1_1 and PC2_2
- can be ignored. The reason for all the `fabs' functions is because the
- signs are usually different.*/
- if( fabs(wcs->pc[1])<ABSOLUTEFLTERROR ) wcs->pc[1]=0.0f;
- if( fabs(wcs->pc[2])<ABSOLUTEFLTERROR ) wcs->pc[2]=0.0f;
- diff=fabs(wcs->pc[0])-fabs(wcs->pc[3]);
- if( fabs(diff/p->pixelscale[0])<RELATIVEFLTERROR )
- wcs->pc[3] = ( (wcs->pc[3] < 0.0f ? -1.0f : 1.0f) * fabs(wcs->pc[0]) );
-
/* Save the output into the proper type and write it. */
if(p->cp.type!=p->output->type)
p->output=gal_data_copy_to_new_type_free(p->output, p->cp.type);
@@ -522,7 +525,7 @@ warp(struct warpparams *p)
gal_threads_dist_in_threads(p->output->size, nt, &indexs, &thrdcols);
- /* Start the convolution. */
+ /* Start the warp. */
if(nt==1)
{
iwp[0].p=p;
diff --git a/configure.ac b/configure.ac
index c169f54..7b589c2 100644
--- a/configure.ac
+++ b/configure.ac
@@ -287,8 +287,6 @@ AM_CONDITIONAL([COND_HASHELP2MAN], [test "x$has_help2man" =
"xyes"])
-
-
# Check libjpeg:
AC_SEARCH_LIBS([jpeg_stdio_dest], [jpeg],
[has_libjpeg=yes], [has_libjpeg=no])
@@ -301,6 +299,18 @@ AM_CONDITIONAL([COND_HASLIBJPEG], [test "x$has_libjpeg" =
"xyes"])
+# Check libtiff:
+AC_SEARCH_LIBS([TIFFOpen], [tiff],
+ [has_libtiff=yes], [has_libtiff=no])
+AS_IF([test "x$has_libtiff" = "xyes"],
+ [AC_DEFINE([HAVE_LIBTIFF], [], [Has libtiff])],
+ [anywarnings=yes])
+AM_CONDITIONAL([COND_HASLIBTIFF], [test "x$has_libtiff" = "xyes"])
+
+
+
+
+
# Check libgit2:
AC_SEARCH_LIBS([git_libgit2_init], [git2],
[has_libgit2=1], [has_libgit2=0])
@@ -344,9 +354,9 @@ AC_DEFINE_UNQUOTED([GAL_CONFIG_GNULIBTOOL_EXEC],
["$gnulibtool_exec"],
-# Check Ghostscript: "-dPDFFitPage" option to Ghostscript, used by
-# ConvertType to convert from EPS to PDF, has been introduced in
-# Ghostscript 9.10. Make sure we have at least that version.
+# Check Ghostscript: "-dPDFFitPage" option to Ghostscript, used by the
+# library to convert from EPS to PDF, has been introduced in Ghostscript
+# 9.10. Make sure we have at least that version.
#
# Below, only when Ghostscript exists, we check its version and only if its
# version is larger than 9.10, does Gnuastro finally assume the existence
@@ -466,6 +476,12 @@ AC_ARG_ENABLE([noisechisel],
[AS_IF([test "x$enable_noisechisel" != xno],
[enable_noisechisel=yes; ayes=true])],
[enable_noisechisel=notset])
+AC_ARG_ENABLE([segment],
+ [AS_HELP_STRING([--enable-segment],
+ [Install Segment and other enabled programs.])],
+ [AS_IF([test "x$enable_segment" != xno],
+ [enable_segment=yes; ayes=true])],
+ [enable_segment=notset])
AC_ARG_ENABLE([statistics],
[AS_HELP_STRING([--enable-statistics],
[Install Statistics and other enabled programs.])],
@@ -501,41 +517,43 @@ AC_ARG_ENABLE([warp],
# disabled).
AS_IF([test $ayes = true ],
[
- AS_IF([test $enable_arithmetic = notset], [enable_arithmetic=no])
- AS_IF([test $enable_buildprog = notset], [enable_buildprog=no])
- AS_IF([test $enable_convertt = notset], [enable_convertt=no])
- AS_IF([test $enable_convolve = notset], [enable_convolve=no])
- AS_IF([test $enable_cosmiccal = notset], [enable_cosmiccal=no])
- AS_IF([test $enable_crop = notset], [enable_crop=no])
- AS_IF([test $enable_fits = notset], [enable_fits=no])
- AS_IF([test $enable_match = notset], [enable_match=no])
- AS_IF([test $enable_mkcatalog = notset], [enable_mkcatalog=no])
- AS_IF([test $enable_mknoise = notset], [enable_mknoise=no])
- AS_IF([test $enable_mkprof = notset], [enable_mkprof=no])
+ AS_IF([test $enable_arithmetic = notset], [enable_arithmetic=no])
+ AS_IF([test $enable_buildprog = notset], [enable_buildprog=no])
+ AS_IF([test $enable_convertt = notset], [enable_convertt=no])
+ AS_IF([test $enable_convolve = notset], [enable_convolve=no])
+ AS_IF([test $enable_cosmiccal = notset], [enable_cosmiccal=no])
+ AS_IF([test $enable_crop = notset], [enable_crop=no])
+ AS_IF([test $enable_fits = notset], [enable_fits=no])
+ AS_IF([test $enable_match = notset], [enable_match=no])
+ AS_IF([test $enable_mkcatalog = notset], [enable_mkcatalog=no])
+ AS_IF([test $enable_mknoise = notset], [enable_mknoise=no])
+ AS_IF([test $enable_mkprof = notset], [enable_mkprof=no])
AS_IF([test $enable_noisechisel = notset], [enable_noisechisel=no])
- AS_IF([test $enable_statistics = notset], [enable_statistics=no])
- AS_IF([test $enable_table = notset], [enable_table=no])
-# AS_IF([test $enable_TEMPLATE = notset], [enable_TEMPLATE=no])
- AS_IF([test $enable_warp = notset], [enable_warp=no])
+ AS_IF([test $enable_segment = notset], [enable_segment=no])
+ AS_IF([test $enable_statistics = notset], [enable_statistics=no])
+ AS_IF([test $enable_table = notset], [enable_table=no])
+# AS_IF([test $enable_TEMPLATE = notset], [enable_TEMPLATE=no])
+ AS_IF([test $enable_warp = notset], [enable_warp=no])
],
[
- AS_IF([test $enable_arithmetic = notset], [enable_arithmetic=yes])
- AS_IF([test $enable_buildprog = notset], [enable_buildprog=yes])
- AS_IF([test $enable_convertt = notset], [enable_convertt=yes])
- AS_IF([test $enable_convolve = notset], [enable_convolve=yes])
- AS_IF([test $enable_cosmiccal = notset], [enable_cosmiccal=yes])
- AS_IF([test $enable_crop = notset], [enable_crop=yes])
- AS_IF([test $enable_fits = notset], [enable_fits=yes])
- AS_IF([test $enable_match = notset], [enable_match=yes])
- AS_IF([test $enable_mkcatalog = notset], [enable_mkcatalog=yes])
- AS_IF([test $enable_mknoise = notset], [enable_mknoise=yes])
- AS_IF([test $enable_mkprof = notset], [enable_mkprof=yes])
+ AS_IF([test $enable_arithmetic = notset], [enable_arithmetic=yes])
+ AS_IF([test $enable_buildprog = notset], [enable_buildprog=yes])
+ AS_IF([test $enable_convertt = notset], [enable_convertt=yes])
+ AS_IF([test $enable_convolve = notset], [enable_convolve=yes])
+ AS_IF([test $enable_cosmiccal = notset], [enable_cosmiccal=yes])
+ AS_IF([test $enable_crop = notset], [enable_crop=yes])
+ AS_IF([test $enable_fits = notset], [enable_fits=yes])
+ AS_IF([test $enable_match = notset], [enable_match=yes])
+ AS_IF([test $enable_mkcatalog = notset], [enable_mkcatalog=yes])
+ AS_IF([test $enable_mknoise = notset], [enable_mknoise=yes])
+ AS_IF([test $enable_mkprof = notset], [enable_mkprof=yes])
AS_IF([test $enable_noisechisel = notset], [enable_noisechisel=yes])
- AS_IF([test $enable_statistics = notset], [enable_statistics=yes])
- AS_IF([test $enable_table = notset], [enable_table=yes])
-# AS_IF([test $enable_TEMPLATE = notset], [enable_TEMPLATE=yes])
- AS_IF([test $enable_warp = notset], [enable_warp=yes])
+ AS_IF([test $enable_segment = notset], [enable_segment=yes])
+ AS_IF([test $enable_statistics = notset], [enable_statistics=yes])
+ AS_IF([test $enable_table = notset], [enable_table=yes])
+# AS_IF([test $enable_TEMPLATE = notset], [enable_TEMPLATE=yes])
+ AS_IF([test $enable_warp = notset], [enable_warp=yes])
]
)
@@ -552,22 +570,23 @@ AS_IF([test "x$has_gnulibtool" = "xno"],
[enable_buildprog=no])
# Make the enable_package values available for the Makefile:
-AM_CONDITIONAL([COND_ARITHMETIC], [test $enable_arithmetic = yes])
-AM_CONDITIONAL([COND_BUILDPROG], [test $enable_buildprog = yes])
-AM_CONDITIONAL([COND_CONVERTT], [test $enable_convertt = yes])
-AM_CONDITIONAL([COND_CONVOLVE], [test $enable_convolve = yes])
-AM_CONDITIONAL([COND_COSMICCAL], [test $enable_cosmiccal = yes])
-AM_CONDITIONAL([COND_CROP], [test $enable_crop = yes])
-AM_CONDITIONAL([COND_FITS], [test $enable_fits = yes])
-AM_CONDITIONAL([COND_MATCH], [test $enable_match = yes])
-AM_CONDITIONAL([COND_MKCATALOG], [test $enable_mkcatalog = yes])
-AM_CONDITIONAL([COND_MKNOISE], [test $enable_mknoise = yes])
-AM_CONDITIONAL([COND_MKPROF], [test $enable_mkprof = yes])
+AM_CONDITIONAL([COND_ARITHMETIC], [test $enable_arithmetic = yes])
+AM_CONDITIONAL([COND_BUILDPROG], [test $enable_buildprog = yes])
+AM_CONDITIONAL([COND_CONVERTT], [test $enable_convertt = yes])
+AM_CONDITIONAL([COND_CONVOLVE], [test $enable_convolve = yes])
+AM_CONDITIONAL([COND_COSMICCAL], [test $enable_cosmiccal = yes])
+AM_CONDITIONAL([COND_CROP], [test $enable_crop = yes])
+AM_CONDITIONAL([COND_FITS], [test $enable_fits = yes])
+AM_CONDITIONAL([COND_MATCH], [test $enable_match = yes])
+AM_CONDITIONAL([COND_MKCATALOG], [test $enable_mkcatalog = yes])
+AM_CONDITIONAL([COND_MKNOISE], [test $enable_mknoise = yes])
+AM_CONDITIONAL([COND_MKPROF], [test $enable_mkprof = yes])
AM_CONDITIONAL([COND_NOISECHISEL], [test $enable_noisechisel = yes])
-AM_CONDITIONAL([COND_STATISTICS], [test $enable_statistics = yes])
-AM_CONDITIONAL([COND_TABLE], [test $enable_table = yes])
-#AM_CONDITIONAL([COND_TEMPLATE], [test $enable_TEMPLATE = yes])
-AM_CONDITIONAL([COND_WARP], [test $enable_warp = yes])
+AM_CONDITIONAL([COND_SEGMENT], [test $enable_segment = yes])
+AM_CONDITIONAL([COND_STATISTICS], [test $enable_statistics = yes])
+AM_CONDITIONAL([COND_TABLE], [test $enable_table = yes])
+#AM_CONDITIONAL([COND_TEMPLATE], [test $enable_TEMPLATE = yes])
+AM_CONDITIONAL([COND_WARP], [test $enable_warp = yes])
@@ -587,6 +606,7 @@ AC_CONFIG_FILES([Makefile
bin/match/Makefile
bin/mkprof/Makefile
bin/mknoise/Makefile
+ bin/segment/Makefile
bin/convertt/Makefile
bin/convolve/Makefile
bin/buildprog/Makefile
@@ -648,17 +668,26 @@ AS_IF([test x$enable_guide_message = xyes],
[
AS_ECHO(["Configuration warning(s):"])
AS_ECHO([])
+
AS_IF([test "x$has_libjpeg" = "xno"],
[AS_ECHO([" - libjpeg, could not be linked with in your library
search path."])
- AS_ECHO([" If JPEG outputs are desired from ConvertType, it
will warn"])
- AS_ECHO([" you and abort with an error."])
+ AS_ECHO([" If JPEG inputs/outputs are requested, the
respective tool will"])
+ AS_ECHO([" inform you and abort with an error."])
+ AS_ECHO([]) ])
+
+ AS_IF([test "x$has_libtiff" = "xno"],
+ [AS_ECHO([" - libtiff, could not be linked with in your library
search path."])
+ AS_ECHO([" If TIFF inputs/outputs are requested, the
respective tool will"])
+ AS_ECHO([" inform you and abort with an error."])
AS_ECHO([]) ])
+
AS_IF([test "x$has_libgit2" = "x0"],
[AS_ECHO([" - libgit2, could not be linked with in your library
search path."])
AS_ECHO([" When present, Git's describe output will be
stored in the"])
AS_ECHO([" output files if Gnuastro's programs were called
within a Git"])
AS_ECHO([" version controlled directory to help in
reproducibility."])
AS_ECHO([]) ])
+
AS_IF([test "x$has_gnulibtool" = "xno"],
[AS_ECHO([" - GNU Libtool, could not be found in your search
path."])
AS_ECHO([" Gnuastro's BuildProgram uses GNU libtool to link
your source code"])
@@ -676,12 +705,14 @@ AS_IF([test x$enable_guide_message = xyes],
AS_ECHO([])
])
])
+
AS_IF([test "x$has_ghostscript" = "xno"],
[AS_ECHO([" - GPL GhostScript version 9.10 or later, with the
executable"])
AS_ECHO([" name \`gs', was not found in your PATH
environment variable."])
- AS_ECHO([" If PDF outputs are desired from ConvertType, it
will abort"])
+ AS_ECHO([" If PDF outputs are desired, the respective tool
it will abort"])
AS_ECHO([" with an EPS output which you can convert to PDF
by other means."])
AS_ECHO([]) ])
+
# The last two scenarios described below are taken from
#
https://unix.stackexchange.com/questions/65700/is-it-safe-to-add-to-my-path-how-come
AS_IF([test "x$path_warning" = "xyes"],
@@ -698,6 +729,7 @@ AS_IF([test x$enable_guide_message = xyes],
AS_ECHO([" installing Gnuastro to learn more about PATH:"])
AS_ECHO([" $ info gnuastro \"Installation directory\""])
AS_ECHO([]) ])
+
AS_ECHO([" All checks related to the warning(s) above will be
skipped."])
AS_ECHO([])
]
diff --git a/doc/Makefile.am b/doc/Makefile.am
index a18dce2..5b496a5 100644
--- a/doc/Makefile.am
+++ b/doc/Makefile.am
@@ -117,6 +117,9 @@ endif
if COND_NOISECHISEL
MAYBE_NOISECHISEL_MAN = man/astnoisechisel.1
endif
+if COND_SEGMENT
+ MAYBE_SEGMENT_MAN = man/astsegment.1
+endif
if COND_STATISTICS
MAYBE_STATISTICS_MAN = man/aststatistics.1
endif
@@ -129,11 +132,12 @@ endif
#if COND_TEMPLATE
# MAYBE_TEMPLATE_MAN = man/astTEMPLATE.1
#endif
-dist_man_MANS = $(MAYBE_ARITHMETIC_MAN) $(MAYBE_BUILDPROG_MAN) \
- $(MAYBE_CONVERTT_MAN) $(MAYBE_CONVOLVE_MAN) $(MAYBE_COSMICCAL_MAN) \
- $(MAYBE_CROP_MAN) $(MAYBE_FITS_MAN) $(MAYBE_MATCH_MAN) $(MAYBE_WARP_MAN) \
- $(MAYBE_MKCATALOG_MAN) $(MAYBE_MKNOISE_MAN) $(MAYBE_MKPROF_MAN) \
- $(MAYBE_NOISECHISEL_MAN) $(MAYBE_STATISTICS_MAN) $(MAYBE_TABLE_MAN)
+dist_man_MANS = $(MAYBE_ARITHMETIC_MAN) $(MAYBE_BUILDPROG_MAN) \
+ $(MAYBE_CONVERTT_MAN) $(MAYBE_CONVOLVE_MAN) $(MAYBE_COSMICCAL_MAN) \
+ $(MAYBE_CROP_MAN) $(MAYBE_FITS_MAN) $(MAYBE_MATCH_MAN) \
+ $(MAYBE_MKCATALOG_MAN) $(MAYBE_MKNOISE_MAN) $(MAYBE_MKPROF_MAN) \
+ $(MAYBE_NOISECHISEL_MAN) $(MAYBE_SEGMENT_MAN) $(MAYBE_STATISTICS_MAN) \
+ $(MAYBE_TABLE_MAN) $(MAYBE_WARP_MAN)
## See if help2man is present or not. When help2man doesn't exist, we don't
@@ -199,6 +203,10 @@ man/astnoisechisel.1: $(top_srcdir)/bin/noisechisel/args.h
$(ALLMANSDEP)
$(MAYBE_HELP2MAN) -n "detect signal in a noisy image" \
--libtool $(toputildir)/noisechisel/astnoisechisel
+man/astsegment.1: $(top_srcdir)/bin/segment/args.h $(ALLMANSDEP)
+ $(MAYBE_HELP2MAN) -n "segmentation based on signal structure" \
+ --libtool $(toputildir)/segment/astsegment
+
man/aststatistics.1: $(top_srcdir)/bin/statistics/args.h $(ALLMANSDEP)
$(MAYBE_HELP2MAN) -n "calculate statistics of a dataset" \
--libtool $(toputildir)/statistics/aststatistics
diff --git a/doc/announce-acknowledge.txt b/doc/announce-acknowledge.txt
index 7e8767c..c962921 100644
--- a/doc/announce-acknowledge.txt
+++ b/doc/announce-acknowledge.txt
@@ -1,11 +1,15 @@
People who's help must be acknowledged in the next release.
+Leindert Boogaard
Nima Dehdilani
Antonio Diaz Diaz
+Lee Kelvin
Guillaume Mahler
+Bertrand Pain
Ole Streicher
Michel Tallon
Juan C. Tello
Éric Thiébaut
Aaron Watkins
Sara Yousefi Taemeh
+Johannes Zabl
diff --git a/doc/gnuastro.texi b/doc/gnuastro.texi
index 81db8c5..1f11cb8 100644
--- a/doc/gnuastro.texi
+++ b/doc/gnuastro.texi
@@ -87,6 +87,9 @@ License''.
* NoiseChisel: (gnuastro)NoiseChisel. Detect signal in noise.
* astnoisechisel: (gnuastro)Invoking astnoisechisel. Options to NoiseChisel.
+* Segment: (gnuastro)Segment. Segment detections based on signal structure.
+* astsegment: (gnuastro)Invoking astsegment. Options to Segment.
+
* Statistics: (gnuastro)Statistics. Get image Statistics.
* aststatistics: (gnuastro)Invoking aststatistics. Options to Statistics.
@@ -425,6 +428,7 @@ Data analysis
* Statistics:: Calculate dataset statistics.
* NoiseChisel:: Detect objects in an image.
+* Segment:: Segment detections based on signal structure.
* MakeCatalog:: Catalog from input and labeled images.
* Match:: Match two datasets.
@@ -448,10 +452,19 @@ NoiseChisel
Invoking NoiseChisel
-* General NoiseChisel options:: General NoiseChisel preprocessing.
+* NoiseChisel input:: NoiseChisel's input options.
* Detection options:: Configure detection in NoiseChisel.
-* Segmentation options:: Configure segmentation in NoiseChisel.
-* NoiseChisel output:: NoiseChisel's output format.
+* NoiseChisel output:: NoiseChisel's output options and format.
+
+Segment
+
+* Invoking astsegment:: Inputs, outputs and options to Segment
+
+Invoking Segment
+
+* Segment input:: Input files and options.
+* Segmentation options:: Parameters of the segmentation process.
+* Segment output:: Outputs of Segment
MakeCatalog
@@ -463,10 +476,10 @@ MakeCatalog
Invoking MakeCatalog
-* MakeCatalog input files:: Specifying the different input files.
-* MakeCatalog general settings:: Options for general column settings.
-* Upper-limit magnitude settings:: Necessary to define upper-limit magnitudes.
-* MakeCatalog output columns:: How to specify the columns in the output.
+* MakeCatalog inputs and basic settings:: Input files and basic settings.
+* Upper-limit settings:: Settings for upper-limit measurements.
+* MakeCatalog output columns:: Available columns in MakeCatalog's output
table.
+* MakeCatalog output file:: File names of MakeCatalog's output table.
Match
@@ -554,10 +567,11 @@ Gnuastro library
* Library data container:: General data container in Gnuastro.
* Dimensions:: Dealing with coordinates and dimensions.
* Linked lists:: Various types of linked lists.
+* Array input output:: Reading and writing images or cubes.
+* Table input output:: Reading and writing table columns.
* FITS files:: Working with FITS data.
+* File input output:: Reading and writing to various file formats.
* World Coordinate System:: Dealing with the world coordinate system.
-* Text files:: Functions to work on Text files.
-* Table input output:: Reading and writing table columns.
* Arithmetic on datasets:: Arithmetic operations on a dataset.
* Tessellation library:: Functions for working on tiles.
* Bounding box:: Finding the bounding box.
@@ -605,6 +619,14 @@ FITS files (@file{fits.h})
* FITS arrays:: Reading and writing FITS images/arrays.
* FITS tables:: Reading and writing FITS tables.
+File input output
+
+* Text files:: Reading and writing from/to plain text files.
+* TIFF files:: Reading and writing from/to TIFF files.
+* JPEG files:: Reading and writing from/to JPEG files.
+* EPS files:: Writing to EPS files.
+* PDF files:: Writing to PDF files.
+
Tessellation library (@file{tile.h})
* Independent tiles:: Work on or check independent tiles.
@@ -1552,12 +1574,19 @@ separate steps and modularized.
@cindex Announcements
@cindex Mailing list: info-gnuastro
-Gnuastro has a dedicated mailing list for making announcements. Anyone
-that is interested can subscribe to this mailing list to stay up to
-date with new releases or when the dependencies (see
-@ref{Dependencies}) have been updated. To subscribe to this list,
-please visit
-@url{https://lists.gnu.org/mailman/listinfo/info-gnuastro}.
+Gnuastro has a dedicated mailing list for making announcements
+(@code{info-gnuastro}). Anyone can subscribe to this mailing list. Anytime
+there is a new stable or test release, an email will be circulated
+there. The email contains a summary of the overall changes along with a
+detailed list (from the @file{NEWS} file). This mailing list is thus the
+best way to stay up to date with new releases, easily learn about the
+updated/new features, or dependencies (see @ref{Dependencies}).
+
+To subscribe to this list, please visit
+@url{https://lists.gnu.org/mailman/listinfo/info-gnuastro}. Traffic (number
+of mails per unit time) in this list is designed to be very low: only a
+handful of mails per year. Previous annoucements are available on
+@url{http://lists.gnu.org/archive/html/info-gnuastro/, its archive}.
@@ -2470,13 +2499,11 @@ The other sections don't have such shortcuts. To
directly access them from
the command-line, you need to tell Info to look into Gnuastro's manual,
then look for the specific section (an unambiguous title is necessary). For
example, if you only want to review/remember NoiseChisel's @ref{Detection
-options} or @ref{Segmentation options}), just run any of these
-commands. Note how case is irrelevant for Info when calling a title in this
-manner.
+options}), just run the following command. Note how case is irrelevant for
+Info when calling a title in this manner.
@example
$ info gnuastro "Detection options"
-$ info gnuastro "segmentation options"
@end example
In general, Info is a wonderful and powerful way to access this whole book
@@ -2691,8 +2718,7 @@ shown. GNU AWK (the most common implementation) comes
with a free and
wonderful @url{https://www.gnu.org/software/gawk/manual/, book} in the same
format as this book which will allow you to master it nicely. Just like
this manual, you can also access GNU AWK's manual on the command-line
-whenever necessary without taking your hands off the keyboard as described
-above.
+whenever necessary without taking your hands off the keyboard.
@end cartouche
This takes us to the second question that you have probably asked yourself
@@ -3058,324 +3084,388 @@ $ rm *.fits
@end example
@noindent
-To detect the objects in the image, we'll run NoiseChisel:
+To detect the signal in the image (separate interesting pixels from noise),
+we'll run NoiseChisel (@ref{NoiseChisel}):
@example
-$ mkdir noisechisel
-$ astnoisechisel flat-ir/xdf-f160w.fits -onoisechisel/xdf-f160w.fits
+$ astnoisechisel flat-ir/xdf-f160w.fits --output=nc-test.fits
@end example
-Read what NoiseChisel writes on the command-line. The curious thing you
-will notice is that while there were more than 3000 pseudo detections to
-find the pseudo-detection S/N, but there were only slightly more than 100
-clumps to find the false clump S/N. We will see what caused this after a
-short review on the output of NoiseChisel.
-
NoiseChisel's output is a single file containing multiple extensions. You
can get basic information about the extensions in a FITS file with
-Gnuastro's Fits program (see @ref{Fits}) as shown below. It contains 6
-extensions and the first (counted as zero) is blank (has no data).
+Gnuastro's Fits program (see @ref{Fits}):
@example
-$ astfits noisechisel/xdf-f160w.fits
+$ astfits nc-test.fits
@end example
-NoiseChisel puts some general information on its outputs in the FITS header
-of the respective extension. To see the full list of keywords, you can
-again use the Fits program like above, but also give it your desired
-extension/HDU. You can also give the extension number (as listed in the
-output above), for example @option{-h2} instead of @option{-hOBJECTS}.
+From the output list, you see NoiseChisel's output contains 5 extensions
+and the first (counted as zero) is blank: it has value of @code{0} in the
+last/size column, showing that it contains no data, it just contains
+meta-data. All of Gnuastro's programs follow this convention of writing no
+data in the first HDU/extension. This allows the first extension to keep
+meta-data about all the extensions and is recommended by the FITS standard,
+see @ref{Fits}.
-@example
-$ astfits noisechisel/xdf-f160w.fits -hOBJECTS
-@end example
+The name of each extension describes its contents: the first
+(@code{INPUT-NO-SKY}) is the Sky-subtracted input that you provided. The
+second (NoiseChisel's main output, @code{DETECTIONS}) has a numeric data
+type of @code{uint8} with only two possible values for all pixels: 0 for
+noise and 1 for signal. The third and fourth (called @code{SKY} and
+@code{SKY_STD}), have the Sky and its standard deviation values of the
+input on a tessellation and were calculated over the un-detected regions.
-@cindex GNU Grep
-The @code{NUMLABS} keyword in NoiseChisel's @code{OBJECTS} extension
-contains the number of objects that was found by NoiseChisel in that
-run. Try to visually find it in the header keywords you saw above.
+@cindex DS9
+@cindex GNOME
+@cindex SAO DS9
+To better understand NoiseChisel's output and the extensions described
+above, let's visually inspect it (here, we'll use SAO DS9). Since the file
+contains multiple related extensions, the easiest way to view all of them
+in DS9 is to open it as a ``Multi-extension data cube'' with the
+@option{-mecube} option as shown below@footnote{You can configure your
+graphic user interface to open DS9 in multi-extension cube mode by default
+when using the GUI (double clicking on the file). If your graphic user
+interface is GNOME (another GNU software, it is most common in GNU/Linux
+operating systems), a full description is given in @ref{Viewing
+multiextension FITS images}}.
+
+@example
+$ ds9 -mecube nc-test.fits -zscale -zoom to fit
+@end example
+
+The buttons and horizontal scroll bar in the ``cube'' window can be used to
+navigate between the extensions. In this mode, all DS9's settings (for
+example zoom or color-bar) will be identical between the extensions. Try
+zooming into to one part and seeing how the galaxies were detected along
+with the Sky and Sky standard deviation values. Just have in mind that
+NoiseChisel's job is @emph{only} detection (separating signal from noise),
+We'll segment this result later.
-To simplify the process, you can pipe the output of the command above into
-@code{grep} (a program for matching lines which is available on almost all
-Unix-like operating systems).
+One good way to see if you have missed any signal is to mask all the
+detected pixels and inspect the noise pixels. For this, you can use
+Gnuastro's Arithmetic program (in particular its @code{where} operator, see
+@ref{Arithmetic operators}). With the command below, all detected pixels
+(in the @code{DETECTIONS} extension) will be set to NaN in the output
+(@file{nc-masked.fits}). To make the command easier to read/write, let's
+just put the file name in a shell variable (@code{img}) first. A shell
+variable's value can be retrieved by adding a @code{$} before its name.
@example
-$ astfits noisechisel/xdf-f160w.fits -hOBJECTS | grep NUMLABS
+$ img=nc-test.fits
+$ astarithmetic $img $img nan where -hINPUT-NO-SKY -hDETECTIONS \
+ --output=nc-masked.fits
@end example
-@cindex GNU Grep
-If you just want the value of the keyword and not the full FITS keyword
-line, you can use AWK. In the example below, AWK will print the third word
-(separated by white space characters) in any line that has a first column
-value of @code{NUMLABS}. You can also try this for the third HDU (called
-@code{CLUMPS}) to see the number of clumps.
+@noindent
+To invert the result (only keep the values of detected pixels), you can
+flip the detected pixel values (from 0 to 1 and vice-versa) by adding a
+@code{not} after the second @code{$img}:
@example
-$ astfits noisechisel/xdf-f160w.fits -h2 \
- | awk '$1=="NUMLABS" @{print $3@}'
+$ astarithmetic $img $img not nan where -hINPUT-NO-SKY -hDETECTIONS \
+ --output=nc-masked.fits
@end example
-Grep and AWK are simple, but very powerful command-line software for
-processing text files. Learning them properly can greatly simplify your
-processing, while improving your creativity, productivity and speed. When
-you get time, it is highly recommended to master them. The most common
-implementation of both is from GNU. Like almost all GNU software, both GNU
-Grep and GNU AWK have wonderful manuals which come with the program for
-free. You don't have to read the whole manual at once, they both start with
-great generic introductions to get you going fast. As described above, you
-can read both manuals or refresh your memory on your command-line with
-these commands:
+NoiseChisel can produce ``Check images'' to help you visualize and inspect
+how each step is completed. You can see all the check images it can produce
+with this command.
@example
-$ info awk
-$ info grep
+$ astnoisechisel --help | grep check
@end example
-@cindex GNOME
-You can now open NoiseChisel's output with SAO DS9 and visually inspect
-it. Just note that since there are multiple extensions, the easiest way to
-view the whole file is to open it as a ``Multi-extension data cube'' with
-the @option{-mecube} option as shown below. If you use GNOME (another GNU
-software, most common graphic user interface in GNU/Linux operating
-systems), please see @ref{Viewing multiextension FITS images} to open DS9
-in multi-extension cube mode by default when using the GUI (double clicking
-on the file).
+Let's see how the quantile threshold (the first step after convolution) has
+been found and applied in our previous run of NoiseChisel:
@example
-$ ds9 -mecube noisechisel/xdf-f160w.fits -zscale -zoom to fit
+$ astnoisechisel flat-ir/xdf-f160w.fits --checkqthresh
@end example
-Using Gnuastro's Fits program, you can also copy a HDU from one FITS file
-to another (for more, see @ref{HDU manipulation}). So if you want to have a
-FITS file with only the detected objects, you can run this command:
+The check images/tables are also multi-extension FITS files. As you see,
+when check datasets are requested, NoiseChisel won't go to the end and
+abort as soon as all its extensions are ready. Try listing the extensions
+with @command{astfits} and then opening them with @command{ds9} as we done
+above. It is @emph{strongly} encouraged to play with the different
+parameters and use the respective check images to see which step is
+affected by your change.
+
+One major factor determining NoiseChisel's the quantile threshold is
+NoiseChisel's ability to identify signal in a tile (the threshold is only
+found on those tiles with no major signal). Therefore the larger the tiles
+are, number-statistics will cause less scatter, therefore helping
+NoiseChisel find the quantile threshold. However, if the tiles are too
+large, there might not be enough over the dataset or they won't be able to
+deal with possible gradients. Let's see what the default (small) tile size
+was with the following command.
@example
-$ astfits noisechisel/xdf-f160w.fits --copy=OBJECTS -oobjects.fits
+$ astnoisechisel -P | grep tilesize
@end example
-One good way to see if you have missed any signal is to mask all the
-detected pixels and inspect the noise pixels. For this, you can use
-Gnuastro's Arithmetic program (in particular its @code{where} operator as
-shown below, see @ref{Arithmetic operators}). With this command, all input
-pixels that have a value larger than zero in the @code{OBJECTS} extension
-will be set to NaN in the output (written in @file{det-masked.fits}). If
-you change the @code{gt} (for ``greater than'') operator to @code{eq} (for
-``equal''), all the un-detected (sky) pixels will be masked and you can see
-the detections.
+Its a 50 by 50 box. Flip back and forth between the @code{CONVOLVED} and
+@code{QTHRESH_ERODE} extensions of the check image to get a feeling of
+which tiles succeeded (have non-blank values)@footnote{The other extensions
+don't matter much for the check here. They show the qthresh values for the
+other thresholds (on the same tiles), followed by the interpolated values
+for all the thresholds and afterwards the smoothed values that are used for
+the next steps. The final extension (@code{QTHRESH-APPLIED}, shows the
+result of applying the erode and no-erode thresholds.}. Since this is a
+relatively large image and we don't have any gradients, let's increase the
+tile size to 100 by 100
@example
-$ astarithmetic noisechisel/xdf-f160w.fits \
- noisechisel/xdf-f160w.fits 0 gt nan where -h1 -h2 \
- --output=nc-masked.fits
+$ astnoisechisel flat-ir/xdf-f160w.fits --tilesize=100,100 \
+ --checkqthresh
@end example
-In some cases, you might want to use a different kernel with NoiseChisel
-(not the default one). To do that, you can use MakeProfiles (see
-@ref{MakeProfiles}) in the following manner to build a 2D Gaussian kernel
-with a FWHM of 3 pixels that extends 5 times the FWHM. This new kernel can
-later be fed into NoiseChisel with the @option{--kernel} option.
+Inspecting the check image, you see that there are now only a handful of
+useful tiles in the central parts. This shows the field is too crowded, and
+we should slightly decrease the tile size for a more robust result that
+also covers more of the dataset. Let's set it to a 75 by 75 pixel box:
@example
-$ astmkprof --kernel=gaussian,3,5 --oversample=1 -okernel-g-3-5.fits
-$ astnoisechisel flat-ir/xdf-f160w.fits --kernel=kernel-g-3-5.fits \
- --output=nc-my-kernel.fits
+$ astnoisechisel flat-ir/xdf-f160w.fits --tilesize=75,75 \
+ --checkqthresh
@end example
-NoiseChisel can produce ``Check images'' to help you visualize how each
-step is completed. You can see all the check images it can produce with
-this command.
+The result seems reasonable now: we have a larger tile size than the
+default value, but less scatter, @emph{and} the tiles cover a sufficiently
+wide area of the dataset. So, we'll use this tile size for the next
+steps. But first, let's clean all the temporary files and make a directory
+for the NoiseChisel outputs:
@example
-$ astnoisechisel --help | grep check
+$ rm *.fits
+$ mkdir nc
+$ astnoisechisel flat-ir/xdf-f160w.fits --tilesize=75,75 \
+ --output=nc/xdf-f160w.fits
+$ astnoisechisel flat-ir/xdf-f105w.fits --tilesize=75,75 \
+ --output=nc/xdf-f105w.fits
@end example
-The check images are also multi-extension FITS files. After each check
-image is produced, open it with ds9 like NoiseChisel's output above and
-flip through the extensions to see each processing in detail. It is
-@emph{strongly} encouraged to play with the different parameters and use
-the respective check images to see which step is affected by your
-change. The three most useful check images are @option{--checkqthresh},
-@option{--checkdetection}, and @option{--checksegmentation}.
-
-We can now get back to the curious situation we noticed after running
-NoiseChisel: the number of false clumps to find an S/N limit was very small
-(given the extent of this image). This is bad, because we use quantiles in
-NoiseChisel and such a small number will result in a relatively large
-scatter. Since this is a segmentation issue, let's see why this happens with
-@option{--checksegmentation}.
+Before continuing with the higher-level processing of this dataset, we'll
+pause to use NoiseChisel's multi-extension output for showing how the Fits
+program can make it very easy to work with this wonderful data container
+(see @ref{Fits}). Let's say you need to copy a HDU/extension from one FITS
+file to another. Try the command below to make an @file{objects.fits} file
+that contains only NoiseChisel's binary detection map. There are similar
+options to conveniently cut or delete HDUs from a FITS file also.
@example
-$ astnoisechisel flat-ir/xdf-f160w.fits --checksegmentation
+$ astfits nc/xdf-f160w.fits --copy=DETECTIONS -odetections.fits
@end example
-To help you get a result faster, when check images are requested,
-NoiseChisel doesn't finish its processing (unless you also call
-@option{--continueaftercheck}). NoiseChisel aborts with an explanation of
-why it stopped without finishing and the file name of the check image that
-it produced. The first five extensions are: the input image, the convolved
-image, the initially labeled detected regions, all the sky region (false)
-clumps, and those that will be used for S/N. The sky clumps are found over
-NoiseChisel's ``large tiles'' independently. When inspecting the fourth
-extension of the check image, it is interesting how NoiseChisel has ignored
-most large tiles and only used the few that we see, mostly on the edge.
-
-The reason that almost all internal large tiles are ignored is that
-galaxies are extended and this is a very deep (and small) image. Thanks to
-the PSF (see @ref{PSF}), no object will have a sharp truncation. We have
-not seen any abrupt break in the light profile of any galaxy: galaxies are
-always larger when we get deeper datasets. Therefore, in a noisy image,
-some light will always be left un-detected. To be less affected by this
-un-detected light, NoiseChisel has the @option{--minskyfrac} option (see
-@ref{General NoiseChisel options}): any tile that has a larger fraction of
-detected pixels will be ignored. So let's see what the default value of
-this option is (recall that with @option{-P}, you can list all the options
-with their values in Gnuastro's programs):
+NoiseChisel puts some general information on its outputs in the FITS header
+of the respective extension. To see the full list of keywords in an
+extension, you can again use the Fits program like above. But instead of
+HDU manipulation options, give it the HDU you are interested in with
+@option{-h}. You can also give the HDU number (as listed in the output
+above), for example @option{-h2} instead of @option{-hDETECTIONS}.
@example
-$ astnoisechisel -P | grep minskyfrac
+$ astfits nc/xdf-f160w.fits -hDETECTIONS
@end example
-@noindent
-Try decreasing this value and re-running NoiseChisel to see the effect on
-the fraction of large tiles that will be used for finding false/sky
-clumps. Play a little with this parameter (give it different values and
-check the result).
+@cindex GNU Grep
+The @code{DETSN} keyword in NoiseChisel's @code{DETECTIONS} extension
+contains the true pseudo-detection signal-to-noise ratio that was found by
+NoiseChisel on the dataset. It is not easy to find it in the middle of all
+the other keywords printed by the command above (especially in files that
+have many more keywords). To fix the problem, you can pipe the output of
+the command above into @code{grep} (a program for matching lines which is
+available on almost all Unix-like operating systems).
@example
-$ astnoisechisel flat-ir/xdf-f160w.fits --minskyfrac=0.5 \
- --checksegmentation
+$ astfits nc/xdf-f160w.fits -hDETECTIONS | grep DETSN
@end example
-The smaller the value to @option{--minskyfrac}, the more probable that
-un-detected light in the wings of galaxies will bias/affect the derived
-false clump S/N. So it is always important to find a good balance between a
-larger @option{--minskyfrac} while having a sufficient number of resulting
-clumps (to avoid scatter).
-
-NoiseChisel doesn't just find the labeled pixels, it also finds the Sky
-value and the Sky standard deviation in the final two extensions of its
-output. To generate a catalog of the colors, we will be using the
-NoiseChisel labeled image from the F160W image. But the Sky and Sky
-standard deviation values for each different filter will also be
-necessary. So we'll run NoiseChisel with our finalized parameters value on
-both filters (you can also put this option's value in a configuration file
-to avoid repeating it).
+@cindex GNU Grep
+If you just want the value of the keyword and not the full FITS keyword
+line, you can use AWK. In the example below, AWK will print the third word
+(separated by white space characters) in any line that has a first column
+value of @code{DETSN}.
@example
-$ astnoisechisel flat-ir/xdf-f105w.fits -onoisechisel/xdf-f105w.fits \
- --minskyfrac=0.5
-$ astnoisechisel flat-ir/xdf-f160w.fits -onoisechisel/xdf-f160w.fits \
- --minskyfrac=0.5
+$ astfits nc/xdf-f160w.fits -h2 | awk '$1=="DETSN" @{print $3@}'
@end example
-Now, we are ready to make a catalog. We want the object and clump labels
-from the F160W image. But the input, Sky and Sky standard deviation images
-should come from each filter. So, we'll run MakeCatalog on NoiseChisel's
-output differently for each filter. When making the F105W catalog, we'll
-use the @option{--objectsfile} and @option{--clumpsfile} options to tell
-MakeCatalog to read the object and clump labels from the F160W NoiseChisel
-output. When these options aren't given, MakeCatalog will look into the
-same input file for object and clump labels.
-
-For both filters, we'll ask for the ID, RA, Dec, Magnitude and
-signal-to-noise ratio (see @ref{Quantifying measurement limits}). To see a
-list of all the parameters that MakeCatalog can measure for you, run it
-with @option{--help} option.
+The main output of NoiseChisel is the binary detection map
+(@code{DETECTIONS} extension), which only has two values of 1 or 0. This is
+useful when studying the noise, but hardly of any use when you actually
+want to study the targets/galaxies in the image, especially in such a deep
+field where the detection map of almost everything is connected. To find
+the galaxies over the detections, we'll use Gnuastro's @ref{Segment}
+program:
@example
-$ mkdir catalog
-
-$ astmkcatalog noisechisel/xdf-f160w.fits --zeropoint=25.94 \
- --ids --ra --dec --magnitude --sn \
- --output=catalog/xdf-f160w.fits
-
-$ astmkcatalog noisechisel/xdf-f105w.fits --zeropoint=26.27 \
- --objectsfile=noisechisel/xdf-f160w.fits \
- --clumpsfile=noisechisel/xdf-f160w.fits \
- --ids --ra --dec --magnitude --sn \
- --output=catalog/xdf-f105w.fits
+$ rm *.fits
+$ mkdir segment
+$ astsegment nc/xdf-f160w.fits -osegment/xdf-f160w.fits
+@end example
+
+Segment's operation is very much like NoiseChisel (in fact, prior to
+version 0.6, it was part of NoiseChisel), for example the output is a
+multi-extension FITS file, it has check images and uses the signal-to-noise
+ratio of undetected regions as a reference. Please have a look at Segment's
+multi-extension output with @command{ds9} to get a good feeling of what it
+has done. Like NoiseChisel, the first extension is the input. The
+@code{CLUMPS} extension shows the true ``clumps'' with values that are
+@mymath{\ge1}, and the diffuse regions labeled as @mymath{-1}. In the
+@code{OBJECTS} extension, we see that the large detections of NoiseChisel
+(that may have contained many galaxies) are now broken up into separate
+labels. see @ref{Segment} for more.
+
+Having localized the regions of interest in the dataset, we are ready to do
+measurements on them with @ref{MakeCatalog}. Besides the IDs, we want to
+measure the Right Ascension (with @option{--ra}), Declination
+(@option{--dec}, magnitude (@option{--magnitude} and signal-to-noise ratio
+(@option{--sn}) of the objects and clumps. The following command will make
+these measurements on Segment's F160W output:
+
+@c Keep the `--zeropoint' on a single line, because later, we'll add
+@c `--valuesfile' in that line also, and it would be more clear if both
+@c catalogs follow the same format.
+@example
+$ mkdir cat
+$ astmkcatalog seg/xdf-f160w.fits --ids --ra --dec --magnitude --sn \
+ --zeropoint=25.94 \
+ --clumpscat --output=cat/xdf-f160w.fits
@end example
-MakeCatalog can also produce catalogs in plain text format. Please run the
-MakeCatalog commands above again and replace the @file{.fits} suffix, in
-the value to @option{--output}, with @file{.txt} (we will also be using the
-text file outputs later).
-
-When a clumps image is also given@footnote{MakeCatalog will look at the
-@code{WCLUMPS} keyword in the objects image to see if it should also use a
-clumps image or not.}, like this example, two catalogs will be made. If you
-asked for a plain text file output, two files will be made with the
-@file{_c.txt} and @file{_o.txt} suffixes. If MakeCatalog's output is
-requested to be FITS, the two catalogs will be in separate extensions of a
-single file. You can inspect the separate extensions with the Fits program
-like before (as shown below). Afterwards, you can inspect the table in each
-extension with Gnuastro's Table program (see @ref{Table}) as shown below.
+@noindent
+From the printed statements on the command-line, you see that MakeCatalog
+read all the extensions in Segment's output for the various measurements it
+needed.
+
+The output of the MakeCatalog command above is a FITS table. The two clump
+and object catalogs are available in the two extensions of the single FITS
+file@footnote{MakeCatalog can also output plain text tables. However, in
+the plain text format you can only have one table per file. Therefore, if
+you also request measurements on clumps, two plain text tables will be
+created (suffixed with @file{_o.txt} and @file{_c.txt}).}. Let's inspect
+the separate extensions with the Fits program like before (as shown
+below). Later, we'll inspect the table in each extension with Gnuastro's
+Table program (see @ref{Table}). Note that we could have used
+@option{-hOBJECTS} and @option{-hCLUMPS} instead of @option{-h1} and
+@option{-h2} respectively.
+
+@example
+$ astfits cat/xdf-f160w.fits # Extension information
+$ asttable cat/xdf-f160w.fits -h1 --info # Objects catalog info.
+$ asttable cat/xdf-f160w.fits -h1 # Objects catalog columns.
+$ asttable cat/xdf-f160w.fits -h2 -i # Clumps catalog info.
+$ asttable cat/xdf-f160w.fits -h2 # Clumps catalog columns.
+@end example
+
+As you see above, when given a specific table (file name and extension),
+Table will print the full contents of all the columns. To see basic
+information about each column (for example name, units and comments),
+simply append a @option{--info} (or @option{-i}).
+
+To print the contents of special column(s), just specify the column
+number(s) (counting from @code{1}) or the column name(s) (if they have
+one). For example, if you just want the magnitude and signal-to-noise ratio
+of the clumps (in @option{-h2}), you can get it with any of the following
+commands
+
+@example
+$ asttable cat/xdf-f160w.fits -h2 -c5,6
+$ asttable cat/xdf-f160w.fits -h2 -c5,SN
+$ asttable cat/xdf-f160w.fits -h2 -c5 -c6
+$ asttable cat/xdf-f160w.fits -h2 -cMAGNITUDE -cSN
+@end example
+
+In the example above, the clumps catalog has two ID columns (one for the
+over-all clump ID and one for the ID of the clump in its host object),
+while the objects catalog only has one ID column. Therefore, the location
+of the magnitude column differs between the object and clumps catalog. So
+if you want to specify the columns by number, you will need to change the
+numbers when viewing the clump and objects catalogs. This is a useful
+advantage of having/using column names@footnote{Column meta-data (including
+a name) can also be specified in plain text tables, see @ref{Gnuastro text
+table format}.}.
@example
-$ astfits catalog/xdf-f105w.fits # Extension information
-$ asttable catalog/xdf-f105w.fits -h1 --info # Objects catalog info.
-$ asttable catalog/xdf-f105w.fits -h1 # Objects catalog columns.
-$ asttable catalog/xdf-f105w.fits -h2 -i # Clumps catalog info.
-$ asttable catalog/xdf-f105w.fits -h2 # Clumps catalog columns.
+$ asttable catalog/xdf-f160w.fits -h1 -c4 -c5
+$ asttable catalog/xdf-f160w.fits -h2 -c5 -c6
@end example
-As you see above, to see the column contents of each table, you can just
-remove the @option{--info} (or @option{-i}) option. If you want to print
-the contents of special column(s), just specify the column number(s)
-(counting from @code{1}, same as output of the command above) or the column
-name(s) (if they have one). For example, if you just want the objects and
-clumps magnitude and signal-to-noise ratio in the F160W filter. You can get
-it with the following commands.
+Finally, the comments in MakeCatalog's output (@code{COMMENT} keywords in
+the FITS headers, or lines starting with @code{#} in plain text) contain
+some important information about the input dataset that can be useful (for
+example pixel area or per-pixel surface brightness limit). For example have
+a look at the output of this command:
@example
-$ asttable catalog/xdf-f160w.fits -h1 -cMAGNITUDE -cSN
-$ asttable catalog/xdf-f160w.fits -h2 -cMAGNITUDE -cSN
+$ astfits cat/xdf-f160w.fits -h1 | grep COMMENT
@end example
-The clumps catalog has two ID columns (one for the over-all clump ID and
-one for the ID of the clump in its host object), the magnitude column
-numbers differ between the object and clumps catalog. So if you want to
-specify the columns by number, you will need to change the numbers when
-viewing the clump and objects catalogs. This is a useful advantage of
-having/using column names.
+To calculate colors, we also need magnitude measurements on the F105W
+filter. However, the galaxy properties might differ between the filters
+(which is the whole purpose behind measuring colors). Also, the noise
+properties and depth of the datasets differ. Therefore, if we simply follow
+the same Segment and MakeCatalog calls above for the F105W filter, we are
+going to get a different number of objects and clumps. Matching the two
+catalogs is possible (for example with @ref{Match}), but the fact that the
+measurements will be done on different pixels, can bias the result. Since
+the Point Spread Function (PSF) of both images is very similar, an accurate
+color calculation can only be done when magnitudes are measured from the
+same pixels on both images.
+
+The F160W image is deeper, thus providing better detection/segmentation,
+and redder, thus observing smaller/older stars and representing more of the
+mass in the galaxies. We will thus use the pixel labels generated on the
+F160W filter, but do the measurements on the F105W filter (using the
+@option{--valuesfile} option) in the command below. Notice how the only
+difference between this call to MakeCatalog and the previous one is
+@option{--valuesfile}, the value given to @code{--zeropoint} and the output
+name.
@example
-$ asttable catalog/xdf-f160w.fits -h1 -c4 -c5
-$ asttable catalog/xdf-f160w.fits -h2 -c5 -c6
+$ astmkcatalog seg/xdf-f160w.fits --ids --ra --dec --magnitude --sn \
+ --valuesfile=nc/xdf-f105w.fits --zeropoint=26.27 \
+ --clumpscat --output=cat/xdf-f105w.fits
@end example
-Finally, the comments in MakeCatalog's output (in FITS headers or lines
-starting with @code{#} in plain text) contain some important information
-about the dataset that can be useful. Open @file{catalog/xdf-f160w_o.txt}
-in a text editor to see them.
+Look into what MakeCatalog printed on the command-line. You can see that
+(as requested) the object and clump labels were taken from the respective
+extensions in @file{seg/xdf-f160w.fits}, while the values and Sky standard
+deviation were done on @file{nc/xdf-f105w.fits}.
Since we used the same labeled image on both filters, the number of rows in
-both catalogs are the same. So, let's measure the colors of the objects in
-this image. We'll merge the two clump catalogs together into one using the
-@code{paste} program on the command-line. The output file will have each
-line of both catalogs merged into a single line.
+both catalogs are the same. The clumps are not affected by the
+hard-to-deblend and low signal-to-noise diffuse regions, they are more
+robust for calculating the colors (compared to objects). Therefore from
+this step onward, we'll continue with clumps.
+
+We can finally calculate the colors of the objects from these two
+datasets. We'll merge them into one table using the @command{paste} program
+on the command-line. But, we only want the magntitude from the F105W
+dataset, so we'll only pull out the @code{MAGNITUDE} and @code{SN}
+column. The output of @command{paste} will have each line of both catalogs
+merged into a single line.
@example
-$ paste catalog/xdf-f160w_c.txt catalog/xdf-f105w_c.txt \
- > xdf-f160w-f105w_c_p.txt
+$ asttable cat/xdf-f160w.fits -h2 > xdf-f160w.txt
+$ asttable cat/xdf-f105w.fits -h2 -cMAGNITUDE,SN > xdf-f105w.txt
+$ paste xdf-f160w.txt xdf-f105w.txt > xdf-f160w-f105w.txt
@end example
-Open @file{xdf-f160w-f105w_c_p.txt} after making it to see how
-@command{paste} has operated. We can now use AWK to find the colors. We'll
-ask AWK to only use lines that don't start with @code{#} and don't have a
-NaN magnitude in the 9th column (F105W magnitude@footnote{Recall that the
-objects and clumps labels were made on the F160W image. On the F105W image,
-there might not be enough signal, so random scatter may give a negative
-total brightness and thus a NaN magnitude.}). We will also ignore columns
-which don't have reliable F105W magnitudes (with a S/N less than
-7@footnote{The value of 7 is taken from the clump S/N threshold in F160W
-(where the clumps were defined).}). For the other lines, AWK will print the
-ID, positional columns and the difference between the respective magnitude
-columns.
+Open @file{xdf-f160w-f105w.txt} to see how @command{paste} has operated. We
+can now use AWK to find the colors. We'll ask AWK to only use lines that
+don't have a NaN magnitude in the 7th column (F105W
+magnitude@footnote{Recall that the objects and clumps labels were made on
+the F160W image. On the F105W image, there might not be enough signal, so
+random scatter may give a negative total brightness and thus a NaN
+magnitude.}). We will also ignore columns which don't have reliable F105W
+measurement (with a S/N less than 7@footnote{The value of 7 is taken from
+the clump S/N threshold in F160W (where the clumps were defined).}). For
+the other lines, AWK will print the IDs, positional columns and the
+difference between the respective magnitude columns.
@example
-$ awk '!/^#/ && $11!="nan" && $12>7 @{print $1, $2, $3, $4, $11-$5@}' \
- xdf-f160w-f105w_c_p.txt > catalog/xdf-f105w-f160w_c.txt
+$ awk '$7!="nan" && $8>7 @{print $1, $2, $3, $4, $7-$5@}' \
+ xdf-f160w-f105w.txt > cat/xdf-f105w-f160w_c.txt
@end example
Gnuastro has a simple program for basic statistical analysis. The command
@@ -3389,14 +3479,15 @@ working on a server (where you may not have graphic
user interface), and
finally, its fast.
@example
-$ aststatistics catalog/xdf-f105w-f160w_c.txt -c5
+$ aststatistics cat/xdf-f105w-f160w_c.txt -c5
@end example
You can later use Gnuastro's Statistics program with the
@option{--histogram} option to build a much more fine-grained histogram as
a table to feed into your favorite plotting program for a much more
-accurate/appealing plot. If you just want a specific measure, for example
-the mean, median and standard deviation, you can ask for them specifically:
+accurate/appealing plot (for example with PGFPlots in @LaTeX{}). If you
+just want a specific measure, for example the mean, median and standard
+deviation, you can ask for them specifically:
@example
$ aststatistics catalog/xdf-f105w-f160w_c.txt -c5 --mean --median --std
@@ -3404,21 +3495,23 @@ $ aststatistics catalog/xdf-f105w-f160w_c.txt -c5
--mean --median --std
Some researchers prefer to have colors in a fixed aperture for all the
objects. The colors we calculated above used a different segmentation map
-for each object. This might not satisfy some science cases. To make a fixed
-aperture catalog, we should make a labeled image which has a fixed label
-for each aperture. That labeled image can be given to MakeCatalog instead
-of NoiseChisel's labeled detection image.
+for each object. This might not satisfy some science cases. So, let's make
+a fixed aperture catalog. To make an catalog from fixed apertures, we
+should make a labeled image which has a fixed label for each aperture. That
+labeled image can be given to MakeCatalog instead of Segment's labeled
+detection image.
@cindex GNU AWK
-We'll use the objects catalog in the F160W catalog we generated before for
-the positions and set the other parameters of each profile to be a fixed
-circle of radius 5 pixels (we want all apertures to be fixed
-after all). AWK is a wonderful tool for such jobs as the command below
-shows.
+To generate the apertures catalog, we'll first read the positions from
+F160W catalog we generated before for the positions and set the other
+parameters of each profile to be a fixed circle of radius 5 pixels (we want
+all apertures to be identical after all).
@example
-$ awk '!/^#/@{print $1, $2, $3, 5, 5, 0, 0, 1, $1, 1@}' \
- catalog/xdf-f160w_c.txt > catalog/apertures.txt
+$ rm *.txt
+$ $ asttable cat/xdf-f160w.fits -h2 -cRA,DEC \
+ | awk '!/^#/@{print NR, $1, $2, 5, 5, 0, 0, 1, NR, 1@}' \
+ > apertures.txt
@end example
We can now feed this catalog into MakeProfiles to build the apertures for
@@ -3431,15 +3524,14 @@ a @emph{magnitude} (in log-scale) of the value given in
that column (what
you would expect when simulating a galaxy for example).
@example
-$ astmkprof catalog/apertures.txt --background=flat-ir/xdf-f160w.fits \
- --clearcanvas --replace --type=int16 --mforflatpix \
+$ astmkprof apertures.txt --background=flat-ir/xdf-f160w.fits \
+ --clearcanvas --replace --type=int16 --mforflatpix \
--mode=wcs
@end example
The first thing you might notice in the printed information is that the
profiles are not built in order. This is because MakeProfiles works in
-parallel and parallel CPU operations are asynchronous. Without
-@option{--replace}, the output is the same in any case. You can try running
+parallel, and parallel CPU operations are asynchronous. You can try running
MakeProfiles with one thread (using @option{--numthreads=1} to see how
order is respected in that case.
@@ -3451,99 +3543,102 @@ are interested, please join us in completing Gnuastro
with added
improvements like this (see task 14750
@footnote{@url{https://savannah.gnu.org/task/index.php?14750}}).
-@file{apertures.fits} labeled image can now be fed input into
-MakeCatalog. Similar to how we used the F160W labels for the F105W catalog:
-the labels don't have to be produced by NoiseChisel. In comparison with the
-previous MakeCatalog call, notice how 1) we have no more clumps image, 2)
-that we have set @option{--objectshdu=1} (since @file{apertures.fits} is
-not the output of NoiseChisel where the labeled image is in the third
-extension), and 3) that we are now using @option{--objid} instead of
-@option{--ids} to avoid warnings that some ID columns are only for clumps.
+We can now feed the labeled @file{apertures.fits} labeled image into
+MakeCatalog instead of Segment's output as shown below. In comparison with
+the previous MakeCatalog call, you will notice that there is no more
+@option{--clumpscat} option, since each aperture is treated as a separate
+object.
@example
-$ astmkcatalog noisechisel/xdf-f105w.fits --zeropoint=26.27 \
- --objectsfile=apertures.fits --objectshdu=1 \
- --objid --ra --dec --magnitude --sn \
- --output=catalog/xdf-f105w-aper.fits
+$ astmkcatalog apertures.fits -h1 --zeropoint=26.27 \
+ --valuesfile=nc/xdf-f105w.fits \
+ --ids --ra --dec --magnitude --sn \
+ --output=cat/xdf-f105w-aper.fits
@end example
-Change the filter name and zeropoint magnitudes and run this command again
-to have the fixed aperture magnitude in the F160W filter also. From this
-point on, you can follow the previous steps to derive the color in a fixed
-aperture.
-
-We will now find some of the objects with the strongest color difference
-and make a cutout to inspect them visually: let's see what the objects with
-a color more than two magnitudes look like.
+This catalog has the same number of rows as the catalog produced from
+clumps, therefore similar to how we found colors, you can compare the
+aperutre and clump magnitudes for example. You can also change the filter
+name and zeropoint magnitudes and run this command again to have the fixed
+aperture magnitude in the F160W filter and measure colors on apertures.
-@cindex AWK
-We'll use the @file{catalog/xdf-f105w-f160w_c.txt} file that we produced
-above. With the command below, all lines with a color value more than 2
-will be put in @file{reddest.txt} and inspect it using @command{cat}.
+@cindex GNU AWK
+Let's find some of the objects with the strongest color difference and make
+a cutout to inspect them visually: let's see what the objects with a color
+more than two magnitudes look like. We'll use the
+@file{cat/xdf-f105w-f160w_c.txt} file that we made above. With the command
+below, all lines with a color more than 1.5 will be put in @file{reddest.txt}
@example
-$ awk '$5>1' catalog/xdf-f105w-f160w_c.txt > reddest.txt
-$ cat reddest.txt
+$ awk '$5>1.5' cat/xdf-f105w-f160w_c.txt > red.txt
@end example
-We can now feed @file{reddest.txt} into Gnuastro's crop to see what these
+We can now feed @file{red.txt} into Gnuastro's crop to see what these
objects look like. To keep things clean, we'll make a directory called
-@file{reddest} and ask Crop to save the crops in this directory. We'll also
-add a @file{-f160w.fits} suffix to the crops.
+@file{crop-red} and ask Crop to save the crops in this directory. We'll
+also add a @file{-f160w.fits} suffix to the crops (to remind us which image
+they came from).
@example
-$ mkdir crop
-$ astcrop --mode=wcs --coordcol=3 --coordcol=4 flat-ir/xdf-f160w.fits \
- --catalog=reddest.txt --width=15/3600,15/3600 \
- --suffix=-f160w.fits --output=crop
+$ mkdir crop-red
+$ astcrop --mode=wcs --coordcol=3 --coordcol=4 flat-ir/xdf-f160w.fits \
+ --catalog=red.txt --width=15/3600,15/3600 \
+ --suffix=-f160w.fits --output=crop-red
@end example
Like the MakeProfiles command above, you might notice that the crops aren't
-made in order. Since each crop is independent of the rest, the crops are
-done in parallel and parallel operations are asynchronous. In the command
-above, change @file{f160w} to @file{f105w} to make the crops in both
-filters.
+made in order. This is because each crop is independent of the rest,
+therefore crops are done in parallel, and parallel operations are
+asynchronous. In the command above, you can change @file{f160w} to
+@file{f105w} to make the crops in both filters.
To view the crops more easily (not having to open ds9 for each image), you
-can convert the FITS crops into the JPEG format.
+can convert the FITS crops into the JPEG format with a shell loop like
+below.
@example
+$ cd crop-red
$ for f in *.fits; do \
astconvertt $f --fluxlow=-0.001 --fluxhigh=0.005 --invert -ojpg; \
done
+$ cd ..
@end example
-The loop above is in series: each file is processed only after the previous
-ones are complete. If you have @url{https://www.gnu.org/software/parallel,
-GNU Parallel}, you can greatly speed up this conversion. GNU Parallel will
-run the separate commands simultaneously on different CPU threads in
-parallel. For more information on efficiently using your threads, see
-@ref{Multi-threaded operations}.
+You can now easily use your general graphic user interface image viewer to
+flip through the images more easily. On GNOME, you can use the ``Eye of
+GNOME'' image viewer (with executable name of @file{eog}). Run the command
+below and by pressing the @key{<SPACE>} key, you can flip through the
+images and compare them visually more easily. Of course, the flux ranges
+have been chosen generically here for seeing the fainter parts. Therefore,
+brighter objects will be fully black.
@example
-$ parallel astconvertt --fluxlow=-0.001 --fluxhigh=0.005 --invert \
- -ojpg ::: *.fits
+$ eog 1-f160w.jpg
@end example
-You can now easily use your general GUI image viewer to flip through the
-images more easily. On GNOME, you can use the ``Eye of GNOME'' image viewer
-(with executable name of @file{eog}). Run the command below and by pressing
-the @key{<SPACE>} key, you can flip through the images and compare them
-visually more easily. Of course, the flux ranges have been chosen
-generically here for seeing the fainter parts. Therefore, brighter objects
-will be fully black.
+@cindex GNU Parallel
+The @code{for} loop above to convert the images will do the job in series:
+each file is converted only after the previous ones are complete. If you
+have @url{https://www.gnu.org/software/parallel, GNU Parallel}, you can
+greatly speed up this conversion. GNU Parallel will run the separate
+commands simultaneously on different CPU threads in parallel. For more
+information on efficiently using your threads, see @ref{Multi-threaded
+operations}. Here is a replacement for the shell @code{for} loop above
+using GNU Parallel.
@example
-$ eog 1-f105w.jpg
+$ cd crop-red
+$ parallel astconvertt --fluxlow=-0.001 --fluxhigh=0.005 --invert \
+ -ojpg ::: *.fits
+$ cd ..
@end example
Another thing that is commonly needed is to visually mark these objects on
-the image. DS9 has ``Region''s for this purpose. You just have to convert
-your catalog into a ``region file'' to feed into DS9. To do that, you can
-use AWK again as shown below.
+the image. DS9 has the ``Region''s concept for this purpose. You just have
+to convert your catalog into a ``region file'' to feed into DS9. To do
+that, you can use AWK again as shown below.
@example
-$ cd ..
$ awk 'BEGIN@{print "# Region file format: DS9 version 4.1"; \
print "global color=green width=2"; \
print "fk5";@} \
@@ -3552,20 +3647,25 @@ $ awk 'BEGIN@{print "# Region file format: DS9 version
4.1"; \
@end example
This region file can be loaded into DS9 with its @option{-regions} option
-as shown below (see above for the other options):
+to display over any image (that has world coordinate system). In the
+example below, we'll open Segment's output and load the regions over all
+the extensions (to see the image and the respective clump):
@example
-$ ds9 -mecube noisechisel/xdf-f160w.fits -zscale -zoom to fit \
+$ ds9 -mecube seg/xdf-f160w.fits -zscale -zoom to fit \
-regions load all reddest.reg
@end example
Finally, if this book or any of the programs in Gnuastro have been useful
-for your research, please cite the respective papers. All Gnuastro programs
-have a @option{--cite} option to help you cite the authors' work more
-easily. For example:
+for your research, please cite the respective papers and share your
+thoughts and suggestions with us (it can be very encouraging). All Gnuastro
+programs have a @option{--cite} option to help you cite the authors' work
+more easily. Just note that it may be necessary to cite additional papers
+for different programs, so please try it out for any program you use.
@example
$ astmkcatalog --cite
+$ astnoisechisel --cite
@end example
@@ -3931,15 +4031,26 @@ headers} for a discussion.
@item libjpeg
@pindex libjpeg
@cindex JPEG format
-libjpeg is only used by ConvertType to read from and write to JPEG
-images. @url{http://www.ijg.org/, libjpeg} is a very basic library
-that provides tools to read and write JPEG images, most of the
-GNU/Linux graphic programs and libraries use it. Therefore you most
+libjpeg is only used by ConvertType to read from and write to JPEG images,
+see @ref{Recognized file formats}. @url{http://www.ijg.org/, libjpeg} is a
+very basic library that provides tools to read and write JPEG images, most
+Unix-like graphic programs and libraries use it. Therefore you most
probably already have it installed.
-@url{http://libjpeg-turbo.virtualgl.org/, libjpeg-turbo} is an
-alternative to libjpeg. It uses SIMD instructions for ARM based
-systems that significantly decreases the processing time of JPEG
-compression and decompression algorithms.
+@url{http://libjpeg-turbo.virtualgl.org/, libjpeg-turbo} is an alternative
+to libjpeg. It uses SIMD instructions for ARM based systems that
+significantly decreases the processing time of JPEG compression and
+decompression algorithms.
+
+@item libtiff
+@pindex libtiff
+@cindex TIFF format
+libtiff is used by ConvertType and the libraries to read TIFF images, see
+@ref{Recognized file formats}. @url{http://www.simplesystems.org/libtiff/,
+libtiff} is a very basic library that provides tools to read and write TIFF
+images, most Unix-like operating system graphic programs and libraries use
+it. Therefore even if you don't have it installed, it must be easily
+available in your package manager.
+
@item GPL Ghostscript
@cindex GPL Ghostscript
@@ -4793,13 +4904,15 @@ after you turn on your computer). Therefore you need
administrator or root
privileges to access or modify them.
@item @file{~/.bash_profile}
-If you are using (GNU) Bash as your shell, the commands in this file are run
-once every time you log in to your account.
+If you are using (GNU) Bash as your shell, the commands in this file are
+run, when you log in to your account @emph{through Bash}. Most commonly
+when you login through the virtual console (where there is no graphic user
+interface).
@item @file{~/.bashrc}
If you are using (GNU) Bash as your shell, the commands here will be run
-each time you start a terminal (for example, when you open your terminal
-emulator in the graphic user interface).
+each time you start a terminal and are already logged in. For example, when
+you open your terminal emulator in the graphic user interface.
@end table
@@ -6144,17 +6257,17 @@ configuration doesn't exist in the running version of
the program.
Here is one example of how this option can be used in conjunction with the
@option{--lastconfig} option. Let's assume that you were satisfied with the
-results of this command: @command{astnoisechisel image.fits
---detquant=0.95} (along with various options set in various configuration
-files). You can save the state of NoiseChisel and reproduce that exact
-result on @file{image.fits} later by following these steps (the the extra
-spaces, and @key{\}, are only for easy readability, if you want to try it
-out, only one space between each token is enough).
+results of this command: @command{astnoisechisel image.fits --snquant=0.95}
+(along with various options set in various configuration files). You can
+save the state of NoiseChisel and reproduce that exact result on
+@file{image.fits} later by following these steps (the the extra spaces, and
+@key{\}, are only for easy readability, if you want to try it out, only one
+space between each token is enough).
@example
$ echo "onlyversion X.XX" > reproducible.conf
$ echo "lastconfig 1" >> reproducible.conf
-$ astnoisechisel image.fits --detquant=0.95 -P \
+$ astnoisechisel image.fits --snquant=0.95 -P \
>> reproducible.conf
@end example
@@ -7669,7 +7782,7 @@ $ ls
ABC01.jpg ABC02.jpg
$ astnoisechisel /mnt/data/DEFproject/DEF01.fits # Prog 2
$ ls
-ABC01.jpg ABC02.jpg DEF01_labeled.fits
+ABC01.jpg ABC02.jpg DEF01_detected.fits
@end example
@@ -8261,20 +8374,17 @@ comments and units for the keyword, see the
@option{--update} option above.
@item -H STR
@itemx --history STR
-Add a @code{HISTORY} keyword to the header. The string given to this
-keyword will be separated into multiple keyword cards if it is longer than
-70 characters. With each run only one value for the @option{--history}
-option will be read. If there are multiple, it is the last one. If there is
-an error, Fits will give a warning and return with a non-zero value, but
-will not stop. To stop as soon as an error occurs, run with
-@option{--quitonerror}.
+Add a @code{HISTORY} keyword to the header with the given value. A new
+@code{HISTORY} keyword will be created for every instance of this
+option. If the string given to this option is longer than 70 characters, it
+will be separated into multiple keyword cards. If there is an error, Fits
+will give a warning and return with a non-zero value, but will not stop. To
+stop as soon as an error occurs, run with @option{--quitonerror}.
@item -c STR
@itemx --comment STR
-Add a @code{COMMENT} keyword to the header. Similar to the explanation for
-@option{--history} above. If there is a writing error, Fits will give a
-warning and return with a non-zero value, but will not stop. To stop as
-soon as an error occurs, run with @option{--quitonerror}.
+Add a @code{COMMENT} keyword to the header with the given value. Similar to
+the explanation for @option{--history} above.
@item -t
@itemx --date
@@ -8285,9 +8395,12 @@ not stop. To stop as soon as an error occurs, run with
@option{--quitonerror}.
@item -p
-@itemx --printall
+@itemx --printallkeys
Print all the keywords in the specified FITS extension (HDU) on the
-command-line.
+command-line. If this option is called along with any of the other keyword
+editing commands, as described above, all other editing commands take
+precedence to this. Therefore, it will print the final keywords after all
+the editing has been done.
@item -Q
@itemx --quitonerror
@@ -8386,6 +8499,26 @@ The file name endings that are recognized as a JPEG file
for input
are: @file{.jpg}, @file{.JPG}, @file{.jpeg}, @file{.JPEG},
@file{.jpe}, @file{.jif}, @file{.jfif} and @file{.jfi}.
+@item TIFF
+@cindex TIFF format
+TIFF (or Tagged Image File Format) was originally designed as a common
+format for scanners in the early 90s and since then it has grown to become
+very general. In many aspects, the TIFF stanard is similar to the FITS
+image standard: it can allow data of many types (see @ref{Numeric data
+types}), and also allows multiple images to be stored in a single file
+(each image in the file is called a `directory' in the TIFF
+standard). However, unlike FITS, it can only store images, it has no
+constructs for tables. Another (inconvenient) difference with the FITS
+standard is that keyword names are stored as numbers, not human-readable
+text.
+
+However, outside of astronomy, because of its support of different numeric
+data types, many fields use TIFF images for accurate (for example 16-bit
+integer or floating point for example) imaging data.
+
+Currently ConvertType can only read TIFF images, if you are interested in
+writing TIFF images, please get in touch with us.
+
@item EPS
@cindex EPS
@cindex PostScript
@@ -8649,13 +8782,22 @@ Input:
@item -h STR/INT
@itemx --hdu=STR/INT
In ConvertType, it is possible to call the HDU option multiple times for
-the different input FITS files (corresponding to different color channels)
-in the same order that they are called on the command-line. Except for the
-fact that multiple calls are possible, this option is identical to the
-common @option{--hdu} in @ref{Input output options}. The number of calls to
-this option cannot be less than the number of input FITS files, but if
-there are more, the extra HDUs will be ignored, note that they will be read
-in the order described in @ref{Configuration file precedence}.
+the different input FITS or TIFF files in the same order that they are
+called on the command-line. Note that in the TIFF standard, one `directory'
+(similar to a FITS HDU) may contain multiple color channels (for example
+when the image is in RGB).
+
+Except for the fact that multiple calls are possible, this option is
+identical to the common @option{--hdu} in @ref{Input output options}. The
+number of calls to this option cannot be less than the number of input FITS
+or TIFF files, but if there are more, the extra HDUs will be ignored, note
+that they will be read in the order described in @ref{Configuration file
+precedence}.
+
+Unlike CFITSIO, libtiff (which is used to read TIFF files) only recognizes
+numbers (counting from zero, similar to CFITSIO) for `directory'
+identification. Hence the concept of names is not defined for the
+directories and the values to this option for TIFF files must be numbers.
@end table
@noindent
@@ -8768,7 +8910,7 @@ flux values, by default it is the opposite.
@itemx --fluxlow=FLT
The minimum flux (pixel value) to display in the output image, any pixel
value below this value will be set to this value in the output. If the
-value to this option is the same as @option{--fluxmax}, then no flux
+value to this option is the same as @option{--fluxhigh}, then no flux
truncation will be applied. Note that when multiple channels are given,
this value is used for all the color channels.
@@ -8887,17 +9029,22 @@ One line examples:
## Get the table column information (name, data type, or units):
$ asttable bintab.fits --information
-## Only print those columns which have a name starting with "MAG_":
-$ asttable bintab.fits --column=/^MAG_/
+## Print columns named RA and DEC, followed by all the columns where
+## the name starts with "MAG_":
+$ asttable bintab.fits --column=RA --column=DEC --column=/^MAG_/
+
+## Similar to above, but with one call to --column (or -c) and writes
+## the columns to a file (with metadata) instead of the command-line.
+$ asttable bintab.fits -cRA,DEC,/^MAG_/ --output=out.txt
## Only print the 2nd column, and the third column multiplied by 5:
-$ asttable bintab.fits | awk '@{print $2, 5*$3@}'
+$ asttable bintab.fits -c2,5 | awk '@{print $1, 5*$2@}'
## Only print rows with a value in the 10th column above 100000:
$ asttable bintab.fits | awk '$10>10e5 @{print@}'
## Sort the output columns by the third column, save output:
-$ asttable bintab.fits | 'sort -k3 > output.txt
+$ asttable bintab.fits | 'sort -nk3 > output.txt
## Convert a plain text table to a binary FITS table:
$ asttable plaintext.txt --output=table.fits --tabletype=fits-binary
@@ -8948,11 +9095,15 @@ last command with all the previously typed columns
present, delete
@cindex GNU AWK
@item -c STR/INT
@itemx --column=STR/INT
-Specify the columns to output, see @ref{Selecting table columns} for a
-thorough explanation on how the value to this option is interpreted. To
-select several output columns, this option can also be called any number
-times in one call to Table. The order of the output columns will be the
-same call order on the command-line.
+Specify the columns to read, see @ref{Selecting table columns} for a
+thorough explanation on how the value to this option is interpreted. There
+are two ways to specify multiple columns: 1) multiple calls to this option,
+2) using a comma between each column specifier in one call to this
+option. These different solutions may be mixed in one call to Table: for
+example, @option{-cRA,DEC -cMAG}, or @option{-cRA -cDEC -cMAG} are both
+equivalent to @option{-cRA -cDEC -cMAG}. The order of the output columns
+will be the same order given to the option or in the configuration files
+(see @ref{Configuration file precedence}).
This option is not mandatory, if no specific columns are requested, all the
input table columns are output. When this option is called multiple times,
@@ -9957,7 +10108,7 @@ viewed in ds9) and 4 pixels along the second FITS
dimension (vertical).
Each pixel will be placed in the center of the box that the mean is
calculated on. If the given width along a dimension is even, then the
center is assumed to be between the pixels (not in the center of a
-pixel). When the pixel is close to the center, the pixels of the box that
+pixel). When the pixel is close to the edge, the pixels of the box that
fall outside the image are ignored. Therefore, on the edge, less points
will be used in calculating the mean.
@@ -9976,6 +10127,83 @@ The median is less susceptible to outliers compared to
the mean. As a
result, after median filtering, the pixel values will be more discontinuous
than mean filtering.
+@item filter-sigclip-mean
+Apply a @mymath{\sigma}-clipped mean filtering onto the input dataset. This
+is very similar to @code{filter-mean}, except that all outliers (identified
+by the @mymath{\sigma}-clipping algorithm) have been removed, see
+@ref{Sigma clipping} for more on the basics of this algorithm. As described
+there, two extra input parameters are necessary for
+@mymath{\sigma}-clipping: the multiple of @mymath{\sigma} and the
+termination critera. @code{filter-sigclip-mean} therefore needs to pop two
+other operands from the stack after the dimensions of the box.
+
+For example the line below uses the same box size as the example of
+@code{filter-mean}. However, all elements in the box that are iteratively
+beyond @mymath{3\sigma} of the distribution's median are removed from the
+final calculation of the mean until the change in @mymath{\sigma} is less
+than @mymath{0.2}.
+
+@example
+$ astarithmetic 3 0.2 5 4 image.fits filter-sigclip-mean
+@end example
+
+The median (which needs a sorted dataset) is necessary for
+@mymath{\sigma}-clipping, therefore @code{filter-sigclip-mean} can be
+significantly slower than @code{filter-mean}. However, if there are strong
+outliers in the dataset that you want to ignore (for example emission lines
+on a spectrum when finding the continuum), this is a much better solution.
+
+@item filter-sigclip-median
+Apply a @mymath{\sigma}-clipped median filtering onto the input
+dataset. This operator and its necessary operands are almost identical to
+@code{filter-sigclip-mean}, except that after @mymath{\sigma}-clipping, the
+median value (which is less affected by outliers than the mean) is added
+back to the stack.
+
+@item connected-components
+@cindex Connected components
+Find the connected components in the input dataset (second popped
+operand). The first popped is the connectivity used in the connected
+components algorithm. The second popped operand is the dataset where
+connected components are to be found. It is assumed to be a binary image
+(with values of 0 or 1). It must have an 8-bit unsigned integer type which
+is the format produced by conditional operators. This operator will return
+a labeled dataset where the non-zero pixels in the input will be labeled
+with a counter (starting from 1).
+
+The connectivity is a number between 1 and the number of dimensions in the
+dataset (inclusive). 1 corresponds to the weakest (symmetric) connectivity
+between elements and the number of dimensions the strongest. For example on
+a 2D image, a connectivity of 1 corresponds to 4-connected neighbors and 2
+corresponds to 8-connected neighbors.
+
+One example usage of this operator can be the identification of regions
+above a certain threshold, as in the command below. With this command,
+Arithmetic will first separate all pixels greater than 100 into a binary
+image (where pixels with a value of 1 are above that value). Afterwards, it
+will label all those that are connected.
+
+@example
+$ astarithmetic in.fits 100 gt 2 connected-components
+@end example
+
+If your input dataset doesn't have a binary type, but you know all its
+values are 0 or 1, you can use the @code{uint8} operator (below) to convert
+it to binary.
+
+@item invert
+Invert an unsigned integer dataset. This is the only operator that ignores
+blank values (which are set to be the maximum values in the unsigned
+integer types).
+
+This is useful in cases where the target(s) has(have) been imaged in
+absorption as raw formats (which are unsigned integer types). With this
+option, the maximum value for the given type will be subtracted from each
+pixel value, thus ``inverting'' the image, so the target(s) can be treated
+as emission. This can be useful when the higher-level analysis
+methods/tools only work on emission (positive skew in the noise, not
+negative).
+
@item lt
Less than: If the second popped (or left operand in infix notation, see
@ref{Reverse polish notation}) value is smaller than the first popped
@@ -12402,6 +12630,7 @@ input dataset (with @ref{MakeCatalog}).
@menu
* Statistics:: Calculate dataset statistics.
* NoiseChisel:: Detect objects in an image.
+* Segment:: Segment detections based on signal structure.
* MakeCatalog:: Catalog from input and labeled images.
* Match:: Match two datasets.
@end menu
@@ -13464,64 +13693,122 @@ one for the Sky standard deviation).
@end table
-@node NoiseChisel, MakeCatalog, Statistics, Data analysis
+@node NoiseChisel, Segment, Statistics, Data analysis
@section NoiseChisel
@cindex Labeling
@cindex Detection
@cindex Segmentation
-Once instrumental signatures are removed from the raw data in the initial
-reduction process (see @ref{Data manipulation}). We are ready to derive
-scientific results out of them. But we can't do anything special with a raw
-dataset, for example an image is just an array of values. Every pixel just
-has one value and its position within the image. Therefore, the first step
-of your high-level analysis will be to classify/label the dataset
-elements/pixels into two classes: signal and noise. This process is
-formally known as @emph{detection}. Afterwards, you want to separate the
-detections into multiple components (for example when two detected regions
-aren't touching, they should be treated independently as two distant
-galaxies for example). This higher level classification of the detections
-is known as @emph{segmentation}. NoiseChisel is Gnuastro's program for
-detection and segmentation.
-
-NoiseChisel works based on a new noise-based approach to signal detection
-and was introduced to the astronomical community in
-@url{https://arxiv.org/abs/1505.01664, Akhlaghi and Ichikawa
-[2015]}. NoiseChisel's primary output is an array (image) with the same
-size as the input but containing labels: those pixels with a label of 0 are
-noise/sky while those pixels with labels larger than 0 are detections
-(separate segments will be given positive integers, starting from 1). For
-more on NoiseChisel's particular output format and its benefits (especially
-in conjunction with @ref{MakeCatalog}), please see
-@url{https://arxiv.org/abs/1611.06387, Akhlaghi [2016]}. The published
-paper cannot under go any updates, but the NoiseChisel software has
-evolved, you can see the major changes in @ref{NoiseChisel changes after
-publication}.
-
-Data is inherently mixed with noise: only mock/simulated datasets are free
-of noise. So this process of separating signal from noise is not
-trivial. In particular, most scientifically interesting astronomical
-targets are faint, can have a large variety of morphologies along with a
-large distribution in brightness and size which are all drowned in a ocean
-of noise. So detection is a uniquely vital aspect of any scientific work
-and even more so for astronomical research. This is such a fundamental step
-that designing of NoiseChisel was the primary motivation behind creating
-Gnuastro: the first generation of Gnuastro's programs were all first part
-of what later became NoiseChisel, afterwards they spinned-off into separate
-programs.
+Once instrumental signatures are removed from the raw data (image) in the
+initial reduction process (see @ref{Data manipulation}). You are naturally
+eager to start answering the scientific questions that motivated the data
+collection. However, the raw dataset/image is just an array of
+values/pixels, that is all! These raw values cannot directly be used to
+answer your scientific questions (for example ``how many galaxies are there
+in the image?''). Therefore, the first high-level step will be to classify,
+or label, the dataset elements (pixels) into two classes: 1) noise, where
+random effects are the major contributor to the value, and 2) signal, where
+non-random factors (for example light from a distant galaxy) are
+present. This classification of a dataset is formally known as
+@emph{detection}.
+
+In an observational/experimental dataset, signal is always burried in
+noise: only mock/simulated datasets are free of noise. Therefore detection,
+or the process of separating signal from noise, determines the number of
+objects you study and the accuracy of any higher-level measurement you do
+on them. Detection is thus the most important step of any analysis and is
+not trivial. In particular, the most scientifically interesting
+astronomical targets are faint, can have a large variety of morphologies,
+along with a large distribution in brightness and size. Therefore when
+noise is significant, detection is the uniquely decisive step of your
+scientific result.
@cindex Erosion
-The name of NoiseChisel is derived from the first thing it does after
-thresholding the dataset: to erode it. In mathematical morphology, erosion
-on pixels can be pictured as carving off boundary pixels. Hence, what
-NoiseChisel does is similar to what a wood chisel or stone chisel do. It is
-just not a hardware, but a software. In fact looking at it as a chisel and
-your dataset as a solid cube of rock will greatly help in best using it:
-with NoiseChisel you literally carve the galaxies/stars/comets out of the
-noise. Try running it with the @option{--checkdetection} option to see each
-step of the carving process on your input dataset. You can then change a
-specific option to carve out your signal out of the noise more
-successfully.
+NoiseChisel is Gnuastro's program for detection of targets that don't have
+a sharp border (almost all astronomical objects). When the targets have a
+sharp edges/border, a simple threshold is enough to separate them from
+noise and each other (if they are not touching). For detection of such
+targets, you can use Gnuastro's Arithmetic program in a command like below
+(assuming the threshold is @code{100}, see @ref{Arithmetic}):
+
+@example
+$ astarithmetic in.fits 100 gt 2 connected-components
+@end example
+
+NoiseChisel uses a new noise-based paradigm for detection of very exteded
+and diffuse targets that are drowned deeply in the ocean of noise. It was
+initially introduced in @url{https://arxiv.org/abs/1505.01664, Akhlaghi and
+Ichikawa [2015]}. The name of NoiseChisel is derived from the first thing
+it does after thresholding the dataset: to erode it. In mathematical
+morphology, erosion on pixels can be pictured as carving-off boundary
+pixels. Hence, what NoiseChisel does is similar to what a wood chisel or
+stone chisel do. It is just not a hardware, but a software. In fact,
+looking at it as a chisel and your dataset as a solid cube of rock will
+greatly help in effectively understanding and optimally using it: with
+NoiseChisel you literally carve your targets out of the noise. Try running
+it with the @option{--checkdetection} option to see each step of the
+carving process on your input dataset.
+
+@cindex Segmentation
+NoiseChisel's primary output is a binary detection map with the same size
+as the input but only with two values: 0 and 1. Pixels that don't harbor
+any detected signal (noise) are given a label (or value) of zero and those
+with a value of 1 have been identified as hosting signal. Segmentation is
+the process of classifing the signal into higher-level constructs, for
+example if you have two separate galaxies in one image, after segmentation,
+they will get separate labels. NoiseChisel is only focused on detection
+(separating signal from noise), to segment the signal (into separate
+galaxies for example), Gnuastro has a separate specialized program
+@ref{Segment}. NoiseChisel's output can be directly/readily fed into
+Segment.
+
+For more on NoiseChisel's output format and its benefits (especially in
+conjunction with @ref{Segment} and later @ref{MakeCatalog}), please see
+@url{https://arxiv.org/abs/1611.06387, Akhlaghi [2016]}. Just note that
+when that paper was published, Segment was not yet spinned-off, and
+NoiseChisel done both detection and segmentation. The output is designed to
+be generic enough to be easily used in any higher-level analysis. If your
+targets are not touching after running NoiseChisel and you aren't
+interested in their sub-structure, you don't need the Segment program at
+all. You can ask NoiseChisel to find the connected pixels in the output
+with the @option{--label} option. In this case, the output won't be a
+binary image any more, the signal will have counters/labels starting from 1
+for each connected group of pixels. You can then directly feed
+NoiseChisel's output into MakeCatalog for measurements over the detections
+and the production of a catalog (see @ref{MakeCatalog}).
+
+Thanks to the published papers mentioned above, there is no need to provide
+a more complete introduction to NoiseChisel in this book. However,
+published papers cannot be updated any more, but the software has
+evolved/changed. The changes since publication are documented in
+@ref{NoiseChisel changes after publication}. Afterwards, in @ref{Invoking
+astnoisechisel}, the details of running NoiseChisel and its options are
+discussed.
+
+As discussed above, detection is the single most important step for your
+scientific result. It is therefore very important to obtain a good
+understanding of NoiseChisel (and afterwards @ref{Segment} and
+@ref{MakeCatalog}). We thus strongly recommend that after reading the
+papers above and the respective sections of Gnuastro's book, you play a
+little with the settings (in the order presented in @ref{Invoking
+astnoisechisel}) on a dataset you are familiar with and inspect all the
+check images (options starting with @option{--check}) to see the effect of
+each parameter.
+
+@ref{General program usage tutorial} is also a good place to get a feeling
+of the modular principle behind Gnuastro's programs and how they are built
+to complement, and build upon, each other. The tutorial culminates in using
+NoiseChisel to detect galaxies and use its outputs to find the galaxy
+colors. Defining colors is a very common process in most
+science-cases. Therefore it is also recommended to (patiently) complete
+that tutorial for optimal usage of NoiseChisel in conjunction with all the
+other Gnuastro programs.
+
+In @ref{NoiseChisel changes after publication}, we'll review the changes in
+NoiseChisel since the publication of @url{https://arxiv.org/abs/1611.06387,
+Akhlaghi [2016]}. We will then review NoiseChisel's input, detection, and
+output options in @ref{NoiseChisel input}, @ref{Detection options}, and
+@ref{NoiseChisel output}.
@menu
* NoiseChisel changes after publication:: Changes to the software after
publication.
@@ -13531,42 +13818,52 @@ successfully.
@node NoiseChisel changes after publication, Invoking astnoisechisel,
NoiseChisel, NoiseChisel
@subsection NoiseChisel changes after publication
-Before using NoiseChisel it is strongly recommended to read
-@url{https://arxiv.org/abs/1505.01664, Akhlaghi and Ichikawa [2015]} to
-gain a good understanding of what it does and how each parameter influences
-the output. Thanks to that paper, there is no need to go into the details
-of the major processing steps. Hence we can just dive into the details of
-running NoiseChisel in @ref{Invoking astnoisechisel}.
-
-However, the paper cannot undergo any further updates, but NoiseChisel will
-evolve: better algorithms or steps will be found, thus options will be
-added or removed. So this book is the final and definitive guide. For a
-more detailed list of changes in each release, please follow the
-@file{NEWS} file. The @file{NEWS} file is in the released Gnuastro tarball
-(see @ref{Release tarball}). You can also view the most recent @file{NEWS}
-file @url{http://git.savannah.gnu.org/cgit/gnuastro.git/plain/NEWS,
-online}@footnote{The online version of the @file{NEWS} file here may
-contain features that have been implemented, but not yet officially
-released as a tarball (see @ref{Downloading the source}). Therefore, please
-be sure to look at the dates and versions above each group of changed
-features and make sure it corresponds to your installed version. It is
-hence recommended to always stay up to date, see @ref{Announcements}).}.
-
-To make the transition form the paper to this book easier (and encourage
-reading the paper), below you can see the major changes since the paper was
-published. First, the options that have been removed are discussed,
-followed by those that have been added.
+NoiseChisel was initially introduced in
+@url{https://arxiv.org/abs/1505.01664, Akhlaghi and Ichikawa [2015]}. It is
+thus strongly recommended to read it for a good understanding of what it
+does and how each parameter influences the output. To help in understanding
+how it works, that paper has a large number of figures showing every step
+on multiple mock and real examples.
+
+However, the paper cannot be updated anymore, but NoiseChisel has evolved
+(and will continue to do so): better algorithms or steps have been found,
+thus options will be added or removed. This book is thus the final and
+definitive guide to NoiseChisel. For a more detailed list of changes in
+each release, please follow the @file{NEWS} file. The @file{NEWS} file is
+present in the released Gnuastro tarball (see @ref{Release tarball}). It is
+also included as a post-script to Gnuastro's announcements, see
+@ref{Announcements}.
+
+The most important change since the publication of that papaer is that
+NoiseChisel is now focused on detection: in spirit with Gnuastro's modular
+design (see @ref{Program design philosophy}), segmentation of the detected
+signal has been spinned-off to Gnuastro's Segment program (see
+@ref{Segment}). Below you can see the major changes since that paper was
+published. First, the removed options/features are discussed, then we
+review the new features that have been added.
@noindent
-Removed options:
+Removed features/options:
@itemize
+
@item
+@option{--skysubtracted}: This option was used to account for the extra
+noise that is added if the Sky value has already been subtracted. However,
+NoiseChisel estimates the Sky standard deviation based on the input data,
+not exposure maps or other theoretical considerations. Therefore the
+standard deviation of the undetected pixels also contains the errors due to
+any previous sky subtraction. This option is therefore no longer present in
+NoiseChisel.
+
@option{--dilate}: In the paper, true detections were dilated for a final
-dig into the noise. However, simple 8-connected dilation can produce boxy
-results which are not realistic and could miss diffuse flux. The final dig
+dig into the noise. However, the simple 8-connected dilation produced boxy
+results which were not realistic and could miss diffuse flux. The final dig
into the noise is now done by ``grow''ing the true detections, similar to
how true clumps were grown, see the description of @option{--detgrowquant}
below and in @ref{Detection options} for more on the new alternative.
+
+@item
+Segmentation has been completely moved to a new program: @ref{Segment}.
@end itemize
@noindent
@@ -13580,8 +13877,9 @@ thresholds (see @ref{Quantifying signal in a tile}).
Until now, NoiseChisel
would convolve an image once and estimate the proper tiles for quantile
estimations on the convolved image. The same convolved image would later be
used for quantile estimation. A larger kernel does increase the skewness
-(and thus difference between the mode and median), however, it disfigures
-the shapes/morphology of the objects.
+(and thus difference between the mode and median, therefore helps in
+detecting the presence signal), however, it disfigures the
+shapes/morphology of the objects.
This new @option{--widekernel} option (and a corresponding @option{--wkhdu}
option to specify its HDU) option are added to solve such cases. When its
@@ -13631,10 +13929,12 @@ of false detections, see the descriptions under this
option in
@node Invoking astnoisechisel, , NoiseChisel changes after publication,
NoiseChisel
@subsection Invoking NoiseChisel
-NoiseChisel will detect and segment signal in noise producing a
-multi-extension labeled image, ready for input into @ref{MakeCatalog} to
-generate a catalog or other processing. The executable name is
-@file{astnoisechisel} with the following general template
+NoiseChisel will detect signal in noise producing a multi-extension dataset
+containing a binary detection map which is the same size as the input. Its
+output can be readily used for input into @ref{Segment}, for higher-level
+segmentation, or @ref{MakeCatalog} to do measurements and generate a
+catalog. The executable name is @file{astnoisechisel} with the following
+general template
@example
$ astnoisechisel [OPTION ...] InputImage.fits
@@ -13644,12 +13944,15 @@ $ astnoisechisel [OPTION ...] InputImage.fits
One line examples:
@example
-## Detect signal in input.fits:
+## Detect signal in input.fits.
$ astnoisechisel input.fits
-## Detect signal assuming input has 4 channels along first dimension
-## and 1 along the second. Also set the regular tile size to 100 along
-## both dimensions:
+## Inspect all the detection steps after changing a parameter.
+$ astnoisechisel input.fits --qthresh=0.4 --checkdetection
+
+## Detect signal assuming input has 4 amplifier channels along first
+## dimension and 1 along the second. Also set the regular tile size
+## to 100 along both dimensions:
$ astnoisechisel --numchannels=4,1 --tilesize=100,100 input.fits
@end example
@@ -13662,10 +13965,16 @@ shares a large set of common operations with other
Gnuastro programs,
mainly regarding input/output, general processing steps, and general
operating modes. To help in a unified experience between all of Gnuastro's
programs, these operations have the same command-line options, see
-@ref{Common options} for a full list. Since the common options are
-thoroughly discussed there, they are no longer reviewed here. You can see
-all the options with a short description on the command-line with the
-@option{--help} option, see @ref{Getting help}.
+@ref{Common options} for a full list/description (they are not repeated
+here).
+
+As in all Gnuastro programs, options can also be given to NoiseChisel in
+configuration files. For a thorough description on Gnuastro's configuration
+file parsing, please see @ref{Configuration files}. All of NoiseChisel's
+options with a short description are also always available on the
+command-line with the @option{--help} option, see @ref{Getting help}. To
+inspect the option values without actually running NoiseChisel, append your
+command with @option{--printparams} (or @option{-P}).
NoiseChisel's input image may contain blank elements (see @ref{Blank
pixels}). Blank elements will be ignored in all steps of NoiseChisel. Hence
@@ -13684,18 +13993,22 @@ be used. For a 2D image, the default kernel is a 2D
Gaussian with a FWHM of
is discussed in Section 3.1.1 of @url{https://arxiv.org/abs/1505.01664,
Akhlaghi and Ichikawa [2015]}. For a 3D cube, it is a Gaussian with FWHM of
1.5 pixels in the first two dimensions and 0.75 pixels in the third
-dimension. See @ref{Convolution kernel} for kernel related options.
+dimension. See @ref{Convolution kernel} for kernel related options. Passing
+@code{none} to @option{--kernel} will disable convolution. On the other
+hand, through the @option{--convolved} option, you may provide an already
+convolved image, see descriptions below for more.
NoiseChisel defines two tessellations over the input (see
@ref{Tessellation}). This enables it to deal with possible gradients in the
input dataset and also significantly improve speed by processing each tile
-on different threads. The tessellation related options are discussed in
-@ref{Processing options}. In particular, NoiseChisel uses two tessellations
-(with everything between them identical except the tile sizes): a
-fine-grained one with smaller tiles (mainly used in detection) and a more
-larger tiled one which is used for multi-threaded processing. The common
+on different threads simultaneously. Tessellation related options are
+discussed in @ref{Processing options}. In particular, NoiseChisel uses two
+tessellations (with everything between them identical except the tile
+sizes): a fine-grained one with smaller tiles (used in thresholding and Sky
+value estimations) and another with larger tiles which is used for
+pseudo-detections over non-detected regions of the image. The common
Tessellation options described in @ref{Processing options} define all
-parameters of both tessellations, only the large tile size for the latter
+parameters of both tessellations. The large tile size for the latter
tessellation is set through the @option{--largetilesize} option. To inspect
the tessellations on your input dataset, run NoiseChisel with
@option{--checktiles}.
@@ -13703,11 +14016,11 @@ the tessellations on your input dataset, run
NoiseChisel with
@cartouche
@noindent
@strong{Usage TIP:} Frequently use the options starting with
-@option{--check}. Depending on what you want to detect in the data, you can
-often play with the parameters/options for a better result than the default
-parameters. You can start with @option{--checkdetection} and
-@option{--checksegmentation} for the main steps. For their full list please
-run:
+@option{--check}. Since the noise properties differ between different
+datasets, you can often play with the parameters/options for a better
+result than the default parameters. You can start with
+@option{--checkdetection} for the main steps. For the full list of
+NoiseChisel's checking options please run:
@example
$ astnoisechisel --help | grep check
@end example
@@ -13759,30 +14072,44 @@ $ astnoisechisel-3d --numchannels=3,3,1 cube.fits
@end example
In the sections below, NoiseChisel's options are classified into three
-general classes to help in easy navigation. @ref{General NoiseChisel
-options} mainly discusses the options relating to input and those that are
-shared in both detection and segmentation. Options to configure the
-detection are described in @ref{Detection options} and @ref{Segmentation
-options} we discuss how you can fine-tune the segmentation of the
-detections. Finally in @ref{NoiseChisel output} the format of NoiseChisel's
-output is discussed. The order of options here follow the same logical
-order that the respective action takes place within NoiseChisel (note that
-the output of @option{--help} is sorted alphabetically).
+general classes to help in easy navigation. @ref{NoiseChisel input} mainly
+discusses the options relating to input and those that are shared in both
+detection and segmentation. Options to configure the detection are
+described in @ref{Detection options} and @ref{Segmentation options} we
+discuss how you can fine-tune the segmentation of the detections. Finally
+in @ref{NoiseChisel output} the format of NoiseChisel's output is
+discussed. The order of options here follow the same logical order that the
+respective action takes place within NoiseChisel (note that the output of
+@option{--help} is sorted alphabetically).
+
+Below, we'll discuss NoiseChisel's options, classified into two general
+classes, to help in easy navigation. @ref{NoiseChisel input} mainly
+discusses the basic options relating to inputs and prior to the detection
+process detection. Afterwards, @ref{Detection options} fully describes
+every configuration parameter (option) related to detection and how they
+affect the final result. The order of options in this section follow the
+logical order within NoiseChisel. On first reading (while you are still new
+to NoiseChisel), it is therefore strongly recommended to read the options
+in the given order below. The output of @option{--printparams} (or
+@option{-P}) also has this order. However, the output of @option{--help} is
+sorted alphabetically. Finally, in @ref{NoiseChisel output} the format of
+NoiseChisel's output is discussed.
+
@menu
-* General NoiseChisel options:: General NoiseChisel preprocessing.
+* NoiseChisel input:: NoiseChisel's input options.
* Detection options:: Configure detection in NoiseChisel.
-* Segmentation options:: Configure segmentation in NoiseChisel.
-* NoiseChisel output:: NoiseChisel's output format.
+* NoiseChisel output:: NoiseChisel's output options and format.
@end menu
-@node General NoiseChisel options, Detection options, Invoking astnoisechisel,
Invoking astnoisechisel
-@subsubsection General NoiseChisel options
+@node NoiseChisel input, Detection options, Invoking astnoisechisel, Invoking
astnoisechisel
+@subsubsection NoiseChisel input
-The options discussed in this section are mainly regarding the input(s),
-output, and some general processing options that are shared between both
-detection and segmentation. Recall that you can always see the full list of
-Gnuastro's options with the @option{--help} option.
+The options here can be used to configure the inputs and output of
+NoiseChisel, along with some general processing options. Recall that you
+can always see the full list of Gnuastro's options with the @option{--help}
+(see @ref{Getting help}), or @option{--printparams} (or @option{-P}) to see
+their values (see @ref{Operating mode options}).
@table @option
@@ -13793,20 +14120,13 @@ File name of kernel to smooth the image before
applying the threshold, see
value of @option{none}.
The first step of NoiseChisel is to convolve/smooth the image and use the
-convolved image in multiple steps during the processing. It will be used to
-define (and later apply) the quantile threshold (see
-@option{--qthresh}). The convolved image is also used to define the clumps
-(see Section 3.2.1 and Figure 8 of @url{https://arxiv.org/abs/1505.01664,
-Akhlaghi and Ichikawa [2015]}).
-
-To build any kernel in one command (for input into this option), you can
-use MakeProfile's @option{--kernel} option, see @ref{MakeProfiles output
-dataset}.
+convolved image in multiple steps including the finding and applying of the
+quantile threshold (see @option{--qthresh}).
-The @option{--kernel} option is not mandatory. If no kernel is provided and
-the input is a 2D image, a 2D Gaussian profile with a FWHM of 2 pixels
-truncated at 5 times the FWHM is used. This choice of the default kernel is
-discussed in Section 3.1.1 of Akhlaghi and Ichikawa [2015].
+The @option{--kernel} option is not mandatory. If not called, a 2D Gaussian
+profile with a FWHM of 2 pixels truncated at 5 times the FWHM is used. This
+choice of the default kernel is discussed in Section 3.1.1 of Akhlaghi and
+Ichikawa [2015].
For a 3D cube, when no file name is given to @option{--kernel}, a Gaussian
with FWHM of 1.5 pixels in the first two dimensions and 0.75 pixels in the
@@ -13818,24 +14138,40 @@ wavelength). The samplings are also different, in the
default case, the
spatial sampling is assumed to be larger than the spectral sampling, hence
a wider FWHM in the spatial directions, see @ref{Sampling theorem}.
+You can use MakeProfiles to build a kernel with any of its recognized
+profile types and parameters. For more details, please see
+@ref{MakeProfiles output dataset}. For example, the command below will make
+a Moffat kernel (with @mymath{\beta=2.8}) with FWHM of 2 pixels truncated
+at 10 times the FWHM.
+
+@example
+$ astmkprof --oversample=1 --kernel=moffat,2,2.8,10
+@end example
+
+Since convolution can be the slowest step of NoiseChisel, for large
+datasets, you can convolve the image once with Gnuastro's Convolve (see
+@ref{Convolve}), and use the @option{--convolved} option to feed it
+directly to NoiseChisel. This can help getting faster results when you are
+playing/testing the higher-level options.
+
@item --khdu=STR
HDU containing the kernel in the file given to the @option{--kernel}
option.
@item --convolved=STR
-Use this file as the convolved image and don't do convolution and ignore
-@option{--kernel}. NoiseChisel will just check the size of the given
+Use this file as the convolved image and don't do convolution (ignore
+@option{--kernel}). NoiseChisel will just check the size of the given
dataset is the same as the input's size. If a wrong image (with the same
size) is given to this option, the results (errors, bugs, and etc) are
un-predictable. So please use this option with care and in a highly
-controlled environment. On such scenario is discussed below.
-
-In almost all situations, as the input gets larger, the single most CPU and
-time consuming step in NoiseChisel is convolution (the first step in its
-processing). The process of testing NoiseChisel for the best parameters in
-a given analysis will involve running NoiseChisel multiple times: to see
-the effect of each new option value. Therefore, once the kernel is
-finalized, re-convolving the input will greatly hinder fast testing of
+controlled environment, for example in the scenario discussed below.
+
+In almost all situations, as the input gets larger, the single most CPU
+(and time) consuming step in NoiseChisel is convolution (the first step in
+its processing). To test NoiseChisel for the best parameters in a given
+analysis, you have to run NoiseChisel multiple times and see the effect of
+each change. Therefore, once the kernel is finalized, re-convolving the
+input on every change will greatly hinder, or discourage, testing of
higher-level parameters. With this option, you can convolve the input image
with your chosen kernel once before running NoiseChisel, then feed it to
NoiseChisel on each test run and thus save valuable time for better/more
@@ -13850,8 +14186,9 @@ will appear on the edges, see @ref{Spatial vs.
Frequency domain}.
Below you can see an example of such a scenario: you want to see how
variation of the growth level (through the @option{--detgrowquant} option)
will affect the final result. Recall that you can ignore all the extra
-spaces, new lines, and `@code{\}' if you are typing in the terminal (in a
-shell script, remove the @code{$} signs at the start of the lines).
+spaces, new lines, and backslash's (`@code{\}') if you are typing in the
+terminal (in a shell script, remove the @code{$} signs at the start of the
+lines).
@example
## Make the kernel to convolve with.
@@ -13870,7 +14207,7 @@ $ for g in 60 65 70 75 80 85 90; do
\
-@item --convolvedhdu=STR
+@item --chdu=STR
The HDU/extension containing the convolved image in the file given to
@option{--convolved}.
@@ -13880,73 +14217,17 @@ File name of a wider kernel to use in estimating the
difference of the mode
and median in a tile (this difference is used to identify the significance
of signal in that tile, see @ref{Quantifying signal in a tile}). This
convolved image is only used for this purpose. Once the mode is found to be
-sufficiently close to the median, the input convolved with the sharper
-kernel (@option{--kernel}) is used to identify the quantile threshold (see
+sufficiently close to the median, the quantile threshold is found on the
+image convolved with the sharper kernel (@option{--kernel}), see
@option{--qthresh}).
-Since this convolution will significantly slow down the processing, it is
-optional. If no image is given, the mode and median will be found on the
-image that is convolved with the dataset in @option{--kernel}.
+Since convolution will significantly slow down the processing, this option
+is optional. If it isn't given, a single convolved image will be used in
+all situations.
-@item --wkhdu=STR
+@item --whdu=STR
HDU containing the kernel file given to the @option{--widekernel} option.
-@item -E
-@itemx --skysubtracted
-If this option is called, it is assumed that the image has already been sky
-subtracted once. Knowing if the sky has already been subtracted once or not
-is very important in estimating the Signal to noise ratio of the detections
-and clumps. In short an extra @mymath{\sigma_{sky}^2} must be added in the
-error (noise or denominator in the Signal to noise ratio) for every flux
-value that is present in the calculation. This can be interpreted as the
-error in measuring that sky value when it was subtracted by any other
-program. See Section 3.3 in @url{https://arxiv.org/abs/1505.01664, Akhlaghi
-and Ichikawa [2015]}) for a complete explanation.
-
-@item -B FLT
-@itemx --minskyfrac=FLT
-Minimum fraction (value between 0 and 1) of sky (undetected) areas in a
-tile for it to be considered in measuring the following detection and
-segmentation properties.
-
-@itemize
-
-@item
-Measuring the Signal to noise ratio of false detections during the false
-detection removal on small tiles.
-
-@item
-Measuring the sky value (average of undetected pixels) on small tiles. Both
-before the removal of false detections and after it.
-
-@item
-Clump Signal to noise ratio in the sky regions of large files.
-
-@end itemize
-
-Because of the PSF and their intrinsic amorphous properties, astronomical
-objects (except cosmic rays) never have a clear cutoff and commonly sink
-into the noise very slowly. Even below the very low thresholds used by
-NoiseChisel. So when a large fraction of the area of one mesh is covered by
-detections, it is very plausible that their faint wings are present in the
-undetected regions (hence causing a bias in any measurement). To get an
-accurate measurement of the above parameters over the tessellation, tiles
-that harbor too many detected regions should be excluded. The used tiles
-are visible in the respective @option{--check} option of the given step.
-
-@item --minnumfalse=INT
-The minimum number of `pseudo-detections' (to identify false initial
-detections) or clumps (to identifying false clumps) found over the
-un-detected regions to identify a Signal-to-Noise ratio threshold.
-
-The Signal to noise ratio (S/N) of false pseudo-detections and clumps in
-each tile is found using the quantile of the S/N distribution of the
-psudo-detections and clumps over the undetected pixels in each mesh. If the
-number of S/N measurements is not large enough, the quantile will not be
-accurate (can have large scatter). For example if you set
-@option{--detquant=0.99} (or the top 1 percent), then it is best to have at
-least 100 S/N measurements.
-
@item -L INT[,INT]
@itemx --largetilesize=INT[,INT]
The size of each tile for the tessellation with the larger tile
@@ -13954,54 +14235,17 @@ sizes. Except for the tile size, all the other
parameters for this
tessellation are taken from the common options described in @ref{Processing
options}. The format is identical to that of the @option{--tilesize} option
that is discussed in that section.
-
-@item --onlydetection
-If this option is called, no segmentation will be done and the output will
-only have four extensions (no clumps extension, see @ref{NoiseChisel
-output}). The second extension of the output is not going to be objects but
-raw detections (a large region will be given one label): labeling is only
-done based on connectivity. The last two extensions of the output will be
-the Sky and its Standard deviation.
-
-This option can result in faster processing when only the noise properties
-of the image are desired for a catalog using another image's labels for
-example. A common case is when you want to measure colors or SEDs in
-several images. Let's say you have images in two colors: A and B. For
-simplicity also assume that they are exactly on the same position in the
-sky with the same pixel scale.
-
-You choose to set A as a reference, so you run the NoiseChisel fully on
-A. Then you run NoiseChisel on B with @option{--onlydetection} since you
-only need the noise properties of B (for the signal to noise column in its
-catalog). You can then run MakeCatalog on A normally, see
-@ref{MakeCatalog}. To run MakeCatalog on B, you simply set the object and
-clump labels images to those that NoiseChisel produced for A, see
-@ref{Invoking astmkcatalog}.
-
-@item --continueaftercheck
-Continue NoiseChisel after any of the options starting with
-@option{--check}. NoiseChisel involves many steps and as a result, there
-are many checks, allowing to inspect the status of the processing. The
-results of each step affect the next steps of processing, so, when you are
-want to check the status of the processing at one step, the time spent to
-complete NoiseChisel is just wasted/distracting time.
-
-To encourage easier experimentation with the option values, when you use
-any of the NoiseChisel options that start with @option{--check},
-NoiseChisel will abort once all the desired check file(s) is (are)
-completed. If you call the @option{--continueaftercheck} option, you can
-disable this behavior and ask NoiseChisel to continue with the rest of the
-processing after completing the check file(s).
-
@end table
-@node Detection options, Segmentation options, General NoiseChisel options,
Invoking astnoisechisel
+@node Detection options, NoiseChisel output, NoiseChisel input, Invoking
astnoisechisel
@subsubsection Detection options
Detection is the process of separating the pixels in the image into two
groups: 1) Signal and 2) Noise. Through the parameters below, you can
customize the detection process in NoiseChisel. Recall that you can always
-see the full list of Gnuastro's options with the @option{--help} option.
+see the full list of Gnuastro's options with the @option{--help} (see
+@ref{Getting help}), or @option{--printparams} (or @option{-P}) to see
+their values (see @ref{Operating mode options}).
@table @option
@@ -14172,6 +14416,24 @@ it is interpreted as tolerance and if it is larger
than one it is assumed
to be the fixed number of iterations. Hence, in the latter case the value
must be an integer.
+@item -B FLT
+@itemx --minskyfrac=FLT
+Minimum fraction (value between 0 and 1) of Sky (undetected) areas in a
+tile. Only tiles with a fraction of undetected pixels (Sky) larger than
+this value will be used to estimate the Sky value. NoiseChisel uses this
+option value twice to estimate the Sky value: after initial detections and
+in the end when false detections have been removed.
+
+Because of the PSF and their intrinsic amorphous properties, astronomical
+objects (except cosmic rays) never have a clear cutoff and commonly sink
+into the noise very slowly. Even below the very low thresholds used by
+NoiseChisel. So when a large fraction of the area of one mesh is covered by
+detections, it is very plausible that their faint wings are present in the
+undetected regions (hence causing a bias in any measurement). To get an
+accurate measurement of the above parameters over the tessellation, tiles
+that harbor too many detected regions should be excluded. The used tiles
+are visible in the respective @option{--check} option of the given step.
+
@item --checkdetsky
Check the initial approximation of the sky value and its standard deviation
in a FITS file ending with @file{_detsky.fits}. With this option,
@@ -14195,8 +14457,8 @@ ratio of the resulting `psudo-detections' are used to
identify true
vs. false detections. See Section 3.1.5 and Figure 7 in Akhlaghi and
Ichikawa (2015) for a very complete explanation.
-@item -i INT
-@itemx --detsnminarea=INT
+@item -m INT
+@itemx --snminarea=INT
The minimum area to calculate the Signal to noise ratio on the
psudo-detections of both the initially detected and undetected
regions. When the area in a psudo-detection is too small, the Signal to
@@ -14205,7 +14467,7 @@ be heavily skewed to the positive. So it is best to
ignore any
psudo-detection that is smaller than this area. Use
@option{--detsnhistnbins} to check if this value is reasonable or not.
-@item --checkdetsn
+@item --checksn
Save the S/N values of the pseudo-detections and dilated detections into
three files ending with @file{_detsn_sky.XXX}, @file{_detsn_det.XXX}, and
@file{_detsn_dilated.XXX}. The @file{.XXX} is determined from the
@@ -14222,8 +14484,18 @@ created. This allows you to inspect the steps leading
to the final quantile
threshold, this behavior (to abort NoiseChisel) can be disabled with
@option{--continueaftercheck}.
+@item --minnumfalse=INT
+The minimum number of `pseudo-detections' over the un-detected regions to
+identify a Signal-to-Noise ratio threshold. The Signal to noise ratio (S/N)
+of false pseudo-detections in each tile is found using the quantile of the
+S/N distribution of the psudo-detections over the undetected pixels in each
+mesh. If the number of S/N measurements is not large enough, the quantile
+will not be accurate (can have large scatter). For example if you set
+@option{--snquant=0.99} (or the top 1 percent), then it is best to have at
+least 100 S/N measurements.
+
@item -c FLT
-@itemx --detquant=FLT
+@itemx --snquant=FLT
The quantile of the Signal to noise ratio distribution of the
psudo-detections in each mesh to use for filling the large mesh grid. Note
that this is only calculated for the large mesh grids that satisfy the
@@ -14278,8 +14550,8 @@ the derived pseudo-detection S/N limit, that detection
will be
discarded. In an ideal/clean noise, a true detection's S/N should be larger
than its constituent pseudo-detections because its area is larger and it
also covers more signal. However, on a false detections (especially at
-lower @option{--detquant} values), the increase in size can cause a
-decrease in S/N below that threshold.
+lower @option{--snquant} values), the increase in size can cause a decrease
+in S/N below that threshold.
This will improve purity and not change completeness (a true detection will
not be discarded). Because a true detection has flux in its vicinity and
@@ -14315,197 +14587,776 @@ have the same pixel size as the input, but with the
@end table
-@node Segmentation options, NoiseChisel output, Detection options, Invoking
astnoisechisel
-@subsubsection Segmentation options
-Segmentation is the process of (possibly) breaking up a detection into
-multiple segments (technically called @emph{objects} and @emph{clumps} in
-NoiseChisel). In deep surveys segmentation becomes particularly important
-because we will be detecting more diffuse flux so galaxy images are going
-to overlap more. It is thus very important to be able separate the pixels
-within a detection.
-
-In NoiseChisel, segmentation is done by first finding the `true' clumps
-over a detection and then expanding those clumps to a certain flux
-limit. True clumps are found in a process very similar to the true
-detections explained in @ref{Detection options}, see
-@url{https://arxiv.org/abs/1505.01664, Akhlaghi and Ichikawa [2015]} for more
-information. If the connections between the grown clumps are weaker than a
-given threshold, the grown clumps are considered to be separate objects.
-@table @option
-@item -m INT
-@itemx --segsnminarea=INT
-The minimum area which a clump in the undetected regions should have in
-order to be considered in the clump Signal to noise ratio measurement. If
-this size is set to a small value, the Signal to noise ratio of false
-clumps will not be accurately found. It is recommended that this value be
-larger than the value to @option{--detsnminarea}. Because the clumps are
-found on the convolved (smoothed) image while the psudo-detections are
-found on the input image. You can use @option{--checkclumpsn} and
-@option{--checksegmentation} to see if your chosen value is reasonable or
-not.
+@node NoiseChisel output, , Detection options, Invoking astnoisechisel
+@subsubsection NoiseChisel output
-@item --checkclumpsn
-Save the S/N values of the clumps into two files ending with
-@file{_clumpsn_sky.XXX} and @file{_clumpsn_det.XXX}. The @file{.XXX} is
-determined from the @option{--tableformat} option (see @ref{Input output
-options}, for example @file{.txt} or @file{.fits}). You can use these to
-inspect the S/N values and their distribution (in combination with the
-@option{--checksegmentation} option to see where the clumps are). You can
-use Gnuastro's @ref{Statistics} to make a histogram of the distribution
-(ready for plotting in a text file, or a crude ASCII-art demonstration on
-the command-line).
+NoiseChisel's output is a multi-extension FITS file. The main
+extension/dataset is a (binary) detection map. It has the same size as the
+input but with only two possible values for all pixels: 0 (for pixels
+identified as noise) and 1 (for those identified as signal/detections). The
+detection map is followed by a Sky and Sky standard deviation dataset
+(which are calculated from the binary image). By default (when
+@option{--rawoutput} isn't called), NoiseChisel will also subtract the Sky
+value from the input and save the sky-subtracted input as the first
+extension in the output.
+
+The name of the output file can be set by giving a value to
+@option{--output} (this is a common option between all programs and is
+therefore discussed in @ref{Input output options}). If @option{--output}
+isn't used, the input name will be suffixed with @file{_detected.fits} and
+used as output, see @ref{Automatic output}. If any of the options starting
+with @option{--check*} are given, NoiseChisel won't complete and will abort
+as soon as the respective check images are created. For more information on
+the different check images, see the description for the @option{--check*}
+options in @ref{Detection options} (this can be disabled with
+@option{--continueaftercheck}).
+
+The last two extensions of the output are the Sky and its Standard
+deviation, see @ref{Sky value} for a complete explanation. They are
+calculated on the tile grid that you defined for NoiseChisel. By default
+these datasets will have the same size as the input, but with all the
+pixels in one tile given one value. To be more space-efficient (keep only
+one pixel per tile), you can use the @option{--oneelempertile} option, see
+@ref{Tessellation}.
-With this option, NoiseChisel will abort as soon as the two tables are
-created. This allows you to inspect the steps leading to the final S/N
-quantile threshold, this behavior can be disabled with
-@option{--continueaftercheck}.
+@cindex GNOME
+To inspect any of NoiseChisel's output files, assuming you use SAO DS9, you
+can configure your Graphic User Interface (GUI) to open NoiseChisel's
+output as a multi-extension data cube. This will allow you to flip through
+the different extensions and visually inspect the results. This process has
+been described for the GNOME GUI (most common GUI in GNU/Linux operating
+systems) in @ref{Viewing multiextension FITS images}.
-@item -g FLT
-@itemx --segquant=FLT
-The quantile of the noise clump Signal to noise ratio distribution. This
-value is used to identify true clumps over the detected regions. You can
-get the full distribution of clumps S/Ns over the undetected areas with the
-@option{--checkclumpsn} option and see them with
-@option{--checksegmentation}.
+NoiseChisel's output configuration options are described in detail below.
-@item -v
-@itemx --keepmaxnearriver
-Keep a clump whose maximum flux is 8-connected to a river pixel. By default
-such clumps over detections are considered to be noise and are removed
-irrespective of their brightness (see @ref{Flux Brightness and
-magnitude}). Over large profiles, that sink into the noise very slowly,
-noise can cause part of the profile (which was flat without noise) to
-become a very large and with a very high Signal to noise ratio. In such
-cases, the pixel with the maximum flux in the clump will be immediately
-touching a river pixel.
+@table @option
+@item --continueaftercheck
+Continue NoiseChisel after any of the options starting with
+@option{--check} (see @ref{Detection options}. NoiseChisel involves many
+steps and as a result, there are many checks, allowing you to inspect the
+status of the processing. The results of each step affect the next steps of
+processing. Therefore, when you want to check the status of the processing
+at one step, the time spent to complete NoiseChisel is just
+wasted/distracting time.
-@item -G FLT
-@itemx --gthresh=FLT
-Threshold (multiple of the sky standard deviation added with the sky) to
-stop growing true clumps. Once true clumps are found, they are set as the
-basis to segment the detected region. They are grown until the threshold
-specified by this option.
+To encourage easier experimentation with the option values, when you use
+any of the NoiseChisel options that start with @option{--check},
+NoiseChisel will abort once its desired extensions have been written. With
+@option{--continueaftercheck} option, you can disable this behavior and ask
+NoiseChisel to continue with the rest of the processing, even after the
+requested check files are complete.
-@item -y INT
-@itemx --minriverlength=INT
-The minimum length of a river between two grown clumps for it to be
-considered in Signal to noise ratio estimations. Similar to
-@option{--segsnminarea} and @option{--detsnminarea}, if the length of the
-river is too short, the Signal to noise ratio can be noisy and
-unreliable. Any existing rivers shorter than this length will be considered
-as non-existent, independent of their Signal to noise ratio. Since the
-clumps are grown on the input image, this value should best be similar to
-the value of @option{--detsnminarea}. Recall that the clumps were defined
-on the convolved image so @option{--segsnminarea} was larger than
-@option{--detsnminarea}.
+@item -l
+@itemx --label
+Run a connected-components algorithm on the finally detected pixels to
+identify which pixels are connected to which. By default the main output is
+a binary dataset with only two values: 0 (for noise) and 1 (for
+signal/detections). See @ref{NoiseChisel output} for more.
+
+The purpose of Noisechisel is to detect targets that are extended and
+diffuse, with outer parts that sink into the noise very gradually (galaxies
+and stars for example). Since NoiseChisel digs down to extremely low
+surface brightness values, many such targets will commonly be detected
+together as a single large body of connected pixels.
+
+To properly separte connected objects, sophisticated segmentation methods
+are commonly necessary on NoiseChisel's output. Gnuastro has the dedicated
+@ref{Segment} program for this job. Since input images are commonly large
+and can take a significant volume, the extra volume necessary to store the
+labels of the connected components in the detection map (which will be
+created with this @option{--label} option, in 32-bit signed integer type)
+can thus be a major waste of space. Since the default output is just a
+binary dataset, an 8-bit unsigned dataset is enough.
+
+The binary output will also encourage users to segment the result
+separately prior to doing higher-level analysis. As an alternative to
+@option{--label}, if you have the binary detection image, you can use the
+@code{connected-components} operator in Gnuastro's Arithmetic program to
+identify regions that are connected with each other. For example with this
+command (assuming NoiseChisel's output is called @file{nc.fits}):
+
+@example
+$ astarithmetic nc.fits connected-components -hDETECTIONS
+@end example
+
+@item --rawoutput
+Don't include the Sky-subtracted input image as the first extension of the
+output. By default, the Sky-subtracted input is put in the first extension
+of the output. The next extensions are NoiseChisel's main outputs described
+above.
-@item -O FLT
-@itemx --objbordersn=FLT
-The maximum Signal to noise ratio of the rivers between two grown clumps in
-order to consider them as separate `objects'. If the Signal to noise ratio
-of the river between two grown clumps is larger than this value, they are
-defined to be part of one `object'. Note that the physical reality of these
-`objects' can never be established with one image, or even multiple images
-from one broad-band filter. Any method we devise to define `object's over a
-detected region is ultimately subjective.
+The extra Sky-subtracted input can be convenient in checking NoiseChisel's
+output and comparing the detection map with the input: visually see if
+everything you expected is detected (reasonable completeness) and that you
+don't have too many false detections (reasonable purity). This visual
+inspection is simplified if you use SAO DS9 to view NoiseChisel's output as
+a multi-extension datacube, see @ref{Viewing multiextension FITS images}.
-Two very distant galaxies or satellites in one halo might lie in the
-same line of sight and be detected as clumps on one detection. On the
-other hand, the connection (through a spiral arm or tidal tail for
-example) between two parts of one galaxy might have such a low surface
-brightness that they are broken up into multiple detections or
-objects. In fact if you have noticed, exactly for this purpose, this is
-the only Signal to noise ratio that the user gives into
-NoiseChisel. The `true' detections and clumps can be objectively
-identified from the noise characteristics of the image, so you don't
-have to give any hand input Signal to noise ratio.
+When you are satisfied with your NoiseChisel configuration (therefore you
+don't need to check on every run), or you want to archive/transfer the
+outputs, or the datasets become large, or you are running NoiseChisel as
+part of a pipeline, this Sky-subtracted input image can be a significant
+burden (take up a large volume). The fact that the input is also noisy,
+makes it hard to compress it efficiently.
-@item --grownclumps
-In the output (see @ref{NoiseChisel output}) store the grown clumps (or
-full detected region if only one clump was present in that detection). By
-default the original clumps are stored as the third extension of the
-output, if this option is called, it is replaced with the grown clump
-labels.
+In such cases, this @option{--rawoutput} can be used to avoid the extra
+sky-subtracted input in the output. It is always possible to easily produce
+the Sky-subtracted dataset from the input (assuming it is in extension
+@code{1} of @file{in.fits}) and the @code{SKY} extension of NoiseChisel's
+output (let's call it @file{nc.fits}) with a command like below (assuming
+NoiseChisel wasn't run with @option{--oneelempertile}, see
+@ref{Tessellation}):
-@item --checksegmentation
-A file with the suffix @file{_seg.fits} will be created. This file keeps
-all the relevant steps in finding true clumps and segmenting the detections
-into multiple objects in various extensions. Having read the paper or the
-steps above. Examining this file can be an excellent guide in choosing the
-best set of parameters. Note that calling this function will significantly
-slow NoiseChisel. In verbose mode (without the @option{--quiet} option, see
-@ref{Operating mode options}) the important steps (along with their
-extension names) will also be reported.
+@example
+$ astarithmetic in.fits nc.fits - -h1 -hSKY
+@end example
+@end table
-With this option, NoiseChisel will abort as soon as the two tables are
-created. This behavior can be disabled with @option{--continueaftercheck}.
+@cartouche
+@noindent
+@cindex Compression
+@strong{Save space:} with the @option{--rawoutput} and
+@option{--oneelempertile}, NoiseChisel's output will only be one binary
+detection map and two much smaller arrays with one value per tile. Since
+none of these have noise they can be compressed very effectively (without
+any loss of data) with exceptionally high compression ratios. This makes it
+easy to archive, or transfer, NoiseChisel's output even on huge
+datasets. To compress it with the most efficient method (take up less
+volume), run the following command:
-@end table
+@cindex GNU Gzip
+@example
+$ gzip --best noisechisel_output.fits
+@end example
+@noindent
+The resulting @file{.fits.gz} file can then be fed into any of Gnuastro's
+programs directly, or viewed in viewers like SAO DS9, without having to
+decompress it separately (they will just take a little longer, because they
+have to internally decompress it before starting).
+@end cartouche
-@node NoiseChisel output, , Segmentation options, Invoking astnoisechisel
-@subsubsection NoiseChisel output
-The default name and directory of the outputs are explained in
-@ref{Automatic output}. NoiseChisel's default output (when none of the
-options starting with @option{--check} or the @option{--output} option
-are called) is one file ending with @file{_labeled.fits}. This file
-has the extensions listed below:
-@enumerate
-@item
-A copy of the input image, a copy is placed here for the following
-reasons:
-@itemize
-@item
-By flipping through the extensions, a user can check how accurate the
-detection and segmentation process was.
-@item
-All the inputs to MakeCatalog (see @ref{MakeCatalog}) are included in
-this one file which makes the running of MakeCatalog after NoiseChisel
-very easy.
-@end itemize
-@item
-The object/detection labels. Each pixel in the input image is given a label
-in this extension, the labels start from one. If the
-@option{--onlydetection} option is given, each large connected part of the
-image has one label. Without that option, this extension is going to show
-the labels of the objects that are found after segmentation. The total
-number of labels is stored as the value to the @code{NOBJS}/@code{NDETS}
-keyword in the header of this extension. This number is also printed in
-verbose mode.
-@item
-The clump labels when @option{--onlydetection} is not called. All the
-pixels in the input image that belong to a true clump are given a positive
-label in this extension. The detected regions that were not a clump are
-given a negative value to clearly identify the sky noise from the diffuse
-detections. The total number of clumps in this image is stored in the
-@code{NCLUMPS} keyword of this extension and printed in verbose output.
-
-If the @option{--grownclumps} option is called, or a value of @code{1}
-is given to it in any of the configuration files, then instead of the
-original clump regions, the grown clumps will be stored in this
-extension. Note that if there is only one clump (or no clumps) over a
-detected region, then the whole detected region is given a label of 1.
-@item
-The final sky value on each pixel. See @ref{Sky value} for a complete
-explanation.
+@node Segment, MakeCatalog, NoiseChisel, Data analysis
+@section Segment
-@item
-Similar to the previous mesh but for the standard deviation on each
-pixel.
+Once signal is separated from noise (for example with @ref{NoiseChisel}),
+you have a binary dataset: each pixel is either signal (1) or noise
+(0). Signal (for example every galaxy in your image) has been ``detected'',
+but all detections have a label of 1. Therefore while we know which pixels
+contain signal, we still can't find out how many galaxies they contain or
+which detected pixels correspond to which galaxy. At the lowest (most
+generic) level, detection is a kind of segmentation (segmenting the the
+whole dataset into signal and noise, see @ref{NoiseChisel}). Here, we'll
+define segmentation only on signal: to separate and find sub-structure
+within the detections.
-@end enumerate
+@cindex Connected component labeling
+If the targets are clearly separated in the dataset (image), a simple
+@emph{connected
+components}@footnote{@url{https://en.wikipedia.org/wiki/Connected-component_labeling}}
+algorithm (very basic segmentation) is enough to separate the regions that
+are touching/connected. This is such a basic and simple form of
+segmentation that Gnuastro's Arithmetic program has an operator for it: see
+@code{connected-components} in @ref{Arithmetic operators}. Assuming the
+binary dataset is called @file{binary.fits}, you can use it with a command
+like this:
+
+@example
+$ astarithmetic binary.fits 2 connected-components
+@end example
+
+@noindent
+You can even do a very basic detection (a threshold, say at value
+@code{100}) @emph{and} segmentation in Arithmetic with a single command
+like below to apply:
+
+@example
+$ astarithmetic in.fits 100 gt 2 connected-components
+@end example
+
+However, in most astronomical situations our targets are not nicely
+separated or have a sharp boundary/edge (for a threshold to suffice): they
+touch (for example merging galaxies), or are simply in the same
+line-of-sight, causing their images to overlap. In particular, when you do
+your detection with NoiseChisel, you will detect signal to very low surface
+brightness limits: deep into the faint wings of galaxies or bright stars
+(which can extend very far and irregularly from their center). Therefore,
+it often happens that several galaxies are detected as one large
+detection. To continue your scientific analysis, a simple connected
+components algorithm will not suffice. It is therefore necessary to do a
+more sophisticated segmentation and break up the detected pixels (even
+those that are touching) into multiple target objects as accurately as
+possible.
+
+Segment will use a detection map and its corresponding dataset to find
+sub-structure over the detected areas and use them for its
+segmentation. Until Gnuastro version 0.6 (released in 2018), Segment was
+part of @ref{NoiseChisel}. Therefore, similar to NoiseChisel, the best
+place to start reading about Segment and understanding what it does (with
+many illustrative figures) is Section 3.2 of
+@url{https://arxiv.org/abs/1505.01664, Akhlaghi and Ichikawa [2015]}.
+
+As a summary, Segment first finds true @emph{clump}s over the
+detections. Clumps are associated with local maxima and extend over the
+neighboring pixels until they reach a local minimum (@emph{river}). Segment
+will use the distribution of clump signal-to-noise ratios over the
+undetected regions as reference to find ``true'' clumps over the
+detections. Using the undetected regions can be disabled by directly giving
+a signal-to-noise ratio to @option{--clumpsnthresh}.
+
+The true clumps are then grown to a certain threshold over the
+detections. Based on the strength of the connections between the grown
+clumps, they are considered parts of one @emph{object} or as separate
+@emph{object}s. See Section 3.2 of Akhlaghi and Ichikawa [2015] (link
+above) for more. Segment's main output are thus two labeled datasets: 1)
+clumps, and 2) objects. See @ref{Segment output} for more.
+
+To start learning about Segment, especially in relation to detection
+(@ref{NoiseChisel}) and measurement (@ref{MakeCatalog}), the recommended
+references are @url{https://arxiv.org/abs/1505.01664, Akhlaghi and Ichikawa
+[2015]} and @url{https://arxiv.org/abs/1611.06387, Akhlaghi [2016]}. Just
+have in mind that Segment became a separate program in 2018 (after those
+papers). Therefore we don't currently need to extend the introduction any
+further and can directly dive into the invocation details in @ref{Invoking
+astsegment}.
+
+Those papers cannot be updated any more but the software will
+evolve. Therefore this book is the definitive reference. Major changes from
+those papers will be documented here when the software starts to diverge
+from them the future, similar what we have already done for NoiseChisel in
+@ref{NoiseChisel changes after publication}.
+
+
+@menu
+* Invoking astsegment:: Inputs, outputs and options to Segment
+@end menu
+
+@node Invoking astsegment, , Segment, Segment
+@subsection Invoking Segment
+
+Segment will identify substructure within the detected regions of an input
+image. Segment's output labels can be directly used for measurements (for
+example with @ref{MakeCatalog}). The executable name is @file{astsegment}
+with the following general template
+
+@example
+$ astsegment [OPTION ...] InputImage.fits
+@end example
+
+@noindent
+One line examples:
+
+@example
+## Segment NoiseChisel's detected regions.
+$ astsegment default-noisechisel-output.fits
+
+## Use a hand-input S/N value for keeping true clumps
+## (avoid finding the S/N using the undetected regions).
+$ astsegment nc-out.fits --clumpsnthresh=10
+
+## Inspect all the segmentation steps after changing a parameter.
+$ astsegment input.fits --snquant=0.9 --checksegmentaion
+
+## Use the fixed value of 0.01 for the input's Sky standard deviation
+## (in the units of the input), and assume all the pixels are a
+## detection (for example a large structure extending over the whole
+## image), and only keep clumps with S/N>10 as true clumps.
+$ astsegment in.fits --std=0.01 --detection=all --clumpsnthresh=10
+@end example
+
+@cindex Gaussian
+@noindent
+If Segment is to do processing (for example you don't want to get help, or
+see the values of each option), at least one input dataset is necessary
+along with detection and error information, either as separate datasets
+(per-pixel) or fixed values, see @ref{Segment input}. Segment shares a
+large set of common operations with other Gnuastro programs, mainly
+regarding input/output, general processing steps, and general operating
+modes. To help in a unified experience between all of Gnuastro's programs,
+these common operations have the same names and defined in @ref{Common
+options}.
+
+As in all Gnuastro programs, options can also be given to Segment in
+configuration files. For a thorough description of Gnuastro's configuration
+file parsing, please see @ref{Configuration files}. All of Segment's
+options with a short description are also always available on the
+command-line with the @option{--help} option, see @ref{Getting help}. To
+inspect the option values without actually running Segment, append your
+command with @option{--printparams} (or @option{-P}).
+
+To help in easy navigation between Segment's options, they are separately
+discussed in the three sub-sections below: @ref{Segment input} discusses
+how you can customize the inputs to Segment. @ref{Segmentation options} is
+devoted to options specific to the high-level segmentation
+process. Finally, in @ref{Segment output}, we'll discuss options that
+affect Segment's output.
+
+@menu
+* Segment input:: Input files and options.
+* Segmentation options:: Parameters of the segmentation process.
+* Segment output:: Outputs of Segment
+@end menu
+
+@node Segment input, Segmentation options, Invoking astsegment, Invoking
astsegment
+@subsubsection Segment input
+
+Besides the input dataset (for example astronomical image), Segment also
+needs to know the Sky standard deviation and the regions of the dataset
+that it should segment. The values dataset is assumed to be Sky subtracted
+by default. If it isn't, you can ask Segment to subtract the Sky internally
+by calling @option{--sky}. For the rest of this discussion, we'll assume it
+is already sky subtracted.
+
+The Sky and its standard deviation can be a single value (to be used for
+the whole dataset) or a separate dataset (for a separate value per
+pixel). If a dataset is used for the Sky and its standard deviation, they
+must either be the size of the input image, or have a single value per tile
+(generated with @option{--oneelempertile}, see @ref{Processing options} and
+@ref{Tessellation}).
+
+The detected regions/pixels can be specified as a detection map (for
+example see @ref{NoiseChisel output}). If @option{--detection=all}, Segment
+won't read any detection map and assume the whole input is a single
+detection. For example when the dataset is fully covered by a large nearby
+galaxy/globular cluster.
+
+When dataset are to be used for any of the inputs, Segment will assume they
+are multiple extensions of a single file by default (when @option{--std} or
+@option{--detection} aren't called). For example NoiseChisel's default
+output @ref{NoiseChisel output}. When the Sky-subtracted values are in one
+file, and the detection and Sky standard deviation are in another, you just
+need to use @option{--detection}: in the absence of @option{--std}, Segment
+will look for both the detection labels and Sky standard deviation in the
+file given to @option{--detection}. Ultimately, if all three are in
+separate files, you need to call both @option{--detection} and
+@option{--std}.
+
+The extensions of the three mandatory inputs can be speicified with
+@option{--hdu}, @option{--dhdu}, and @option{--stdhdu}. For a full
+discussion on what to give to these options, see the description of
+@option{--hdu} in @ref{Input output options}. To see their default values
+(along with all the other options), run Segment with the
+@option{--printparams} (or @option{-P}) option. Just recall that in the
+absence of @option{--detection} and @option{--std}, all three are assumed
+to be in the same file. If you only want to see Segment's default values
+for HDUs on your system, run this command:
+
+@example
+$ astsegment -P | grep hdu
+@end example
+
+By default Segment will convolve the input with a kernel to improve the
+signal-to-noise ratio of true peaks. If you already have the convolved
+input dataset, you can pass it directly to Segment for faster processing
+(using the @option{--convolved} and @option{--chdu} options). Just don't
+forget that the convolved image must also be Sky-subtracted before calling
+Segment. If a value/file is given to @option{--sky}, the convolved values
+will also be Sky subtracted internally. Alternatively, if you prefer to
+give a kernel (with @option{--kernel} and @option{--khdu}), Segment can do
+the convolution internally. To disable convolution, use
+@option{--kernel=none}.
+
+@table @option
+
+@item --sky=STR/FLT
+The Sky value(s) to subtract from the input. This option can either be
+given a constant number or a file name containing a dataset (multiple
+values, per pixel or per tile). By default, Segment will assume the input
+dataset is Sky subtracted, so this option is not mandatory.
+
+If the value can't be read as a number, it is assumed to be a file
+name. When the value is a file, the extension can be specified with
+@option{--skyhdu}. When its not a single number, the given dataset must
+either have the same size as the output or the same size as the
+tessellation (so there is one pixel per tile, see @ref{Tessellation}).
+
+When this option is given, its value(s) will be subtraced from the input
+and the (optional) convolved dataset (given to @option{--convolved}) prior
+to starting the segmentation process.
+
+@item --skyhdu=STR/INT
+The HDU/extension containing the Sky values. This is mandatory when the
+value given to @option{--sky} is not a number. Please see the description
+of @option{--hdu} in @ref{Input output options} for the different ways you
+can identify a special extension.
+
+@item --std=STR/FLT
+The Sky standard deviation value(s) corresponding to the input. The value
+can either be a constant number or a file name containing a dataset
+(multiple values, per pixel or per tile). The Sky standard deviation is
+mandatory for Segment to operate.
+
+If the value can't be read as a number, it is assumed to be a file
+name. When the value is a file, the extension can be specified with
+@option{--skyhdu}. When its not a single number, the given dataset must
+either have the same size as the output or the same size as the
+tessellation (so there is one pixel per tile, see @ref{Tessellation}).
+
+When this option is not called, Segment will assume the standard deviation
+is a dataset and in a HDU/extension (@option{--stdhdu}) of another one of
+the input file(s). If a file is given to @option{--detection}, it will
+assume that file contains the standard deviation dataset, otherwise, it
+will look into input filename (the main argument, without any option).
+
+@item --stdhdu=INT/STR
+The HDU/extension containing the Sky standard deviation values, when the
+value given to @option{--std} is a file name. Please see the description of
+@option{--hdu} in @ref{Input output options} for the different ways you can
+identify a special extension.
+
+@item --variance
+The input Sky standard deviation value/dataset is actually variance. When
+this option is called, the square root of input Sky standard deviation (see
+@option{--std}) is used internally, not its raw value(s).
+
+@item -d STR
+@itemx --detection=STR
+Detection map to use for segmentation. If given a value of @option{all},
+Segment will assume the whole dataset must be segmented, see below. If a
+detection map is given, the extension can be specified with
+@option{--dhdu}. If not given, Segment will assume the desired
+HDU/extension is in the main input argument (input file specified with no
+option).
+
+Segmentation (clumps or objects) will only take place over the non-zero
+pixels of this detection map. The dataset must have the same size as the
+input image. Only datasets with an integer type are acceptable for the
+labeled image, see @ref{Numeric data types}. If your detection map only has
+integer values, but it is stored in a floating point container, you can use
+Gnuastro's Arithmetic program (see @ref{Arithmetic}) to convert it to an
+integer container, like the example below:
+
+@example
+$ astarithmetic float.fits int32 --output=int.fits
+@end example
+
+It may happen that the whole input dataset is covered by signal, for
+example when working on parts of the Andromeda galaxy, or nearby globular
+clusters (that cover the whole field of view). In such cases, segmentation
+is necessary over the complete dataset, not just specific regions
+(detections). By default Segment will first use the undetected regions as a
+reference to find the proper signal-to-noise ratio of ``true'' clumps (give
+a purity level specified with @option{--snquant}). Therefore, in such
+scenarios you also need to manually give a ``true'' clump signal-to-noise
+ratio with the @option{--clumpsnthresh} option to disable looking into the
+undetected regions, see @ref{Segmentation options}. In such cases, is
+possible to make a detection map that only has the value @code{1} for all
+pixels (for example using @ref{Arithmetic}), but for convenience, you can
+also use @option{--detection=all}.
+
+@item --dhdu
+The HDU/extension containing the detection map given to
+@option{--detection}. Please see the description of @option{--hdu} in
+@ref{Input output options} for the different ways you can identify a
+special extension.
+
+@item -k STR
+@itemx --kernel=STR
+The kernel used to convolve the input image. The usage of this option is
+identical to NoiseChisel's @option{--kernel} option (@ref{NoiseChisel
+input}). Please see the descriptions there for more. To disable
+convolution, you can give it a value of @option{none}.
+
+@item --khdu
+The HDU/extension containing the kernel used for convolution. For
+acceptable values, please see the description of @option{--hdu} in
+@ref{Input output options}.
+
+@item --convolved
+The convolved image to avoid internal convolution by Segment. The usage of
+this option is identical to NoiseChisel's @option{--convolved} option
+(@ref{NoiseChisel input}). Please see the descriptions there for more.
+
+@item --chdu
+The HDU/extension containing the convolved image (given to
+@option{--convolved}). For acceptable values, please see the description of
+@option{--hdu} in @ref{Input output options}.
+
+@item -L INT[,INT]
+@itemx --largetilesize=INT[,INT]
+The size of the large tiles to use for identifying the clump S/N threshold
+over the undetected regions. The usage of this option is identical to
+NoiseChisel's @option{--largetilesize} option (@ref{NoiseChisel
+input}). Please see the descriptions there for more.
+
+The undetected regions can be a significant fraction of the dataset and
+finding clumps requires sorting of the desired regions, which can be
+slow. To speed up the processing, Segment finds clumps in the undetected
+regions over separate large tiles. This allows it to have to sort a much
+smaller set of pixels and also to treat them independently and in
+parallel. Both these issues greatly speed it up. Just be sure to not
+decrease the large tile sizes too much (less than 100 pixels in each
+dimension). It is important for them to be much larger than the clumps.
+
+@end table
+
+
+@node Segmentation options, Segment output, Segment input, Invoking astsegment
+@subsubsection Segmentation options
+
+The options below can be used to configure every step of the segmentation
+process in the Segment program. For a more complete explanation (with
+figures to demonstrate each step), please see Section 3.2 of
+@url{https://arxiv.org/abs/1505.01664, Akhlaghi and Ichikawa [2015]}, and
+also @ref{Segment}. By default, Segment will follow the procedure described
+in the paper to find the S/N threshold based on the noise properties. This
+can be disabled by directly giving a trustable signal-to-noise ratio to the
+@option{--clumpsnthresh} option.
+
+Recall that you can always see the full list of Gnuastro's options with the
+@option{--help} (see @ref{Getting help}), or @option{--printparams} (or
+@option{-P}) to see their values (see @ref{Operating mode options}).
+
+@table @option
+
+@item -B FLT
+@itemx --minskyfrac=FLT
+Minimum fraction (value between 0 and 1) of Sky (undetected) areas in a
+large tile. Only (large) tiles with a fraction of undetected pixels (Sky)
+greater than this value will be used for finding clumps. The clumps found
+in the undetected areas will be used to estimate a S/N threshold for true
+clumps. Therefore this is an important option (to decrease) in crowded
+fields. Operationally, this is almost identical to NoiseChisel's
+@option{--minskyfrac} option (@ref{Detection options}). Please see the
+descriptions there for more.
+
+@item -m INT
+@itemx --snminarea=INT
+The minimum area which a clump in the undetected regions should have in
+order to be considered in the clump Signal to noise ratio measurement. If
+this size is set to a small value, the Signal to noise ratio of false
+clumps will not be accurately found. It is recommended that this value be
+larger than the value to NoiseChisel's @option{--snminarea}. Because the
+clumps are found on the convolved (smoothed) image while the
+psudo-detections are found on the input image. You can use
+@option{--checksn} and @option{--checksegmentation} to see if your chosen
+value is reasonable or not.
+
+@item --checksn
+Save the S/N values of the clumps into two files ending with
+@file{_clumpsn_sky.XXX} and @file{_clumpsn_det.XXX}. The @file{.XXX} is
+determined from the @option{--tableformat} option (see @ref{Input output
+options}, for example @file{.txt} or @file{.fits}). You can use these to
+inspect the S/N values and their distribution (in combination with the
+@option{--checksegmentation} option to see where the clumps are). You can
+use Gnuastro's @ref{Statistics} to make a histogram of the distribution
+(ready for plotting in a text file, or a crude ASCII-art demonstration on
+the command-line).
+
+With this option, NoiseChisel will abort as soon as the two tables are
+created. This allows you to inspect the steps leading to the final S/N
+quantile threshold, this behavior can be disabled with
+@option{--continueaftercheck}.
+
+@item --minnumfalse=INT
+The minimum number of clumps over undetected (Sky) regions to identify the
+requested Signal-to-Noise ratio threshold. Operationally, this is almost
+identical to NoiseChisel's @option{--minnumfalse} option (@ref{Detection
+options}). Please see the descriptions there for more.
+
+@item -c FLT
+@itemx --snquant=FLT
+The quantile of the signal-to-noise ratio distribution of clumps in
+undetected regions, used to define true clumps. After identifying all the
+usable clumps in the undetected regions of the dataset, the given quantile
+of their signal-to-noise ratios is used to define a the signal-to-noise
+ratio of a ``true'' clump. Effectively, this can be seen as an inverse
+p-value measure. See Figure 9 and Section 3.2.1 of
+@url{https://arxiv.org/abs/1505.01664, Akhlaghi and Ichikawa [2015]} for a
+complete explanation. The full distribution of clump signal-to-noise ratios
+over the undetected areas can be saved into a table with @option{--checksn}
+option and visually inspected with @option{--checksegmentation}.
+
+@item -v
+@itemx --keepmaxnearriver
+Keep a clump whose maximum flux is 8-connected to a river pixel. By default
+such clumps over detections are considered to be noise and are removed
+irrespective of their brightness (see @ref{Flux Brightness and
+magnitude}). Over large profiles, that sink into the noise very slowly,
+noise can cause part of the profile (which was flat without noise) to
+become a very large and with a very high Signal to noise ratio. In such
+cases, the pixel with the maximum flux in the clump will be immediately
+touching a river pixel.
+
+@item -s FLT
+@itemx --clumpsnthresh=FLT
+The signal-to-noise threshold for true clumps. If this option is given,
+then the segmentation options above will be ignored and the given value
+will be directly used to identify true clumps over the detections. This can
+be useful if you have a large dataset with similar noise properties. You
+can find a robust signal-to-noise ratio based on a (sufficiently large)
+smaller portion of the dataset. Afterwards, with this option, you can speed
+up the processing on the whole dataset. Other scenarios where this option
+may be useful is when, the image might not contain enough/any Sky regions.
+
+@item -G FLT
+@itemx --gthresh=FLT
+Threshold (multiple of the sky standard deviation added with the sky) to
+stop growing true clumps. Once true clumps are found, they are set as the
+basis to segment the detected region. They are grown until the threshold
+specified by this option.
+
+@item -y INT
+@itemx --minriverlength=INT
+The minimum length of a river between two grown clumps for it to be
+considered in signal-to-noise ratio estimations. Similar to
+@option{--snminarea}, if the length of the river is too short, the
+signal-to-noise ratio can be noisy and unreliable. Any existing rivers
+shorter than this length will be considered as non-existent, independent of
+their Signal to noise ratio. The clumps are grown on the input image,
+therefore this value can be smaller than the value given to
+@option{--snminarea}. Recall that the clumps were defined on the convolved
+image so @option{--snminarea} should be larger.
+
+@item -O FLT
+@itemx --objbordersn=FLT
+The maximum Signal to noise ratio of the rivers between two grown clumps in
+order to consider them as separate `objects'. If the Signal to noise ratio
+of the river between two grown clumps is larger than this value, they are
+defined to be part of one `object'. Note that the physical reality of these
+`objects' can never be established with one image, or even multiple images
+from one broad-band filter. Any method we devise to define `object's over a
+detected region is ultimately subjective.
+
+Two very distant galaxies or satellites in one halo might lie in the
+same line of sight and be detected as clumps on one detection. On the
+other hand, the connection (through a spiral arm or tidal tail for
+example) between two parts of one galaxy might have such a low surface
+brightness that they are broken up into multiple detections or
+objects. In fact if you have noticed, exactly for this purpose, this is
+the only Signal to noise ratio that the user gives into
+NoiseChisel. The `true' detections and clumps can be objectively
+identified from the noise characteristics of the image, so you don't
+have to give any hand input Signal to noise ratio.
+
+@item --checksegmentation
+A file with the suffix @file{_seg.fits} will be created. This file keeps
+all the relevant steps in finding true clumps and segmenting the detections
+into multiple objects in various extensions. Having read the paper or the
+steps above. Examining this file can be an excellent guide in choosing the
+best set of parameters. Note that calling this function will significantly
+slow NoiseChisel. In verbose mode (without the @option{--quiet} option, see
+@ref{Operating mode options}) the important steps (along with their
+extension names) will also be reported.
+
+With this option, NoiseChisel will abort as soon as the two tables are
+created. This behavior can be disabled with @option{--continueaftercheck}.
+
+@end table
+
+@node Segment output, , Segmentation options, Invoking astsegment
+@subsubsection Segment output
+
+The main output of Segment are two label datasets (with integer types,
+separating the dataset's elements into different classes) with
+HDU/extension names of @code{CLUMPS} and @code{OBJECTS}. For a complete
+definition of clumps and objects, please see Section 3.2 of
+@url{https://arxiv.org/abs/1505.01664, Akhlaghi and Ichikawa [2015]} and
+@ref{Segmentation options}.
+
+The clumps are ``true'' local maxima and their surrounding pixels until a
+local minimum (caused by noise fluctuations, or another ``true''
+clump). Therefore it may happen that some of the input detections aren't
+covered by clumps at all (very diffuse objects without any strong peak),
+while some objects may contain many clumps. Even in those that have clumps,
+there will be regions that are too diffuse. The diffuse regions (within the
+input detected regions) are given a negative label (-1) to help you
+separate them from the undetected regions (with a value of zero).
+
+Each clump is labeled with respect to its host object. Therefore, if an
+object has three clumps for example, the clumps within it have labels 1, 2
+and 3. As a result, if an initial detected region has multiple objects,
+each with a single clump, all the clumps will have a label of 1. The total
+number of clumps in the dataset is stored in the @code{NCLUMPS} keyword of
+the @code{CLUMPS} extension and printed in the verbose output of Segment
+(when @option{--quiet} is not called).
+
+The @code{OBJECTS} extension of the output will give a positive
+counter/label to every detected pixel in the input. As described in
+Akhlaghi and Ichikawa [2015], the true clumps are grown until a certain
+threshold. If the grown clumps touch other clumps and the connection is
+strong enough, they are considered part of the same @emph{object}. Once
+objects (grown clumps) are identified, they are grown to cover the whole
+detected area.
+
+@cindex DS9
+@cindex SAO DS9
+By default, besides the @code{CLUMPS} and @code{OBJECTS} extensions,
+Segment's output will also contain the (technically redundant) input
+dataset and the sky standard deviation dataset (if it wasn't a constant
+number). This can help in visually inspecting the result when viewing the
+images as a ``Multi-extension data cube'' in SAO DS9 for example (see
+@ref{Viewing multiextension FITS images}). You can simply flip through the
+exetensions and see the same region of the image and its corresponding
+clumps/object labels. It also makes it easy to feed the output (as one
+file) into MakeCatalog when you intend to make a catalog afterwards (see
+@ref{MakeCatalog}. To remove these redundant extensions from the output
+(for example when designing a pipeline), you can use
+@option{--rawoutput}.
+
+The @code{OBJECTS} and @code{CLUMPS} extensions can be used as input into
+@ref{MakeCatalog} to generate a catalog for higher-level analysis. If you
+want to treat each clump separately, you can give a very large value (or
+even a NaN, which will always fail) to the @option{--gthresh} option (for
+example @code{--gthresh=1e10} or @code{--gthresh=nan}), see
+@ref{Segmentation options}. The options to configure the output of Segment
+are listed below:
+
+@table @option
+@item --continueaftercheck
+Don't abort Segment after producing the check image(s). The usage of this
+option is identical to NoiseChisel's @option{--continueaftercheck} option
+(@ref{NoiseChisel input}). Please see the descriptions there for more.
+
+@item --grownclumps
+In the output @code{CLUMPS} extension, store the grown clumps. If a
+detected region contains no clumps or only one clump, then it will be fully
+given a label of @code{1} (no negative valued pixels).
+
+@item --rawoutput
+Only write the @code{CLUMPS} and @code{OBJECTS} datasets in the output
+file. Without this option (by default), the first and last extensions of
+the output will the Sky-subtracted input dataset and the Sky standard
+deviation dataset (if it wasn't a number). When the datasets are small,
+these redundant extensions can make it convenient to inspect the results
+visually or feed the output to @ref{MakeCatalog} for
+measurements. Ultimately both the input and Sky standard deviation datasets
+are redundant (you had them before running Segment). When the inputs are
+large/numerous, these extra dataset can be a burden.
+@end table
+
+@cartouche
+@noindent
+@cindex Compression
+@strong{Save space:} with the @option{--rawoutput}, Segment's output will
+only be two labeled datasets (only containing integers). Since they have no
+noise, such datasets can be compressed very effectively (without any loss
+of data) with exceptionally high compression ratios. You can use the
+following command to compress it with the best ratio:
+
+@cindex GNU Gzip
+@example
+$ gzip --best segment_output.fits
+@end example
+
+@noindent
+The resulting @file{.fits.gz} file can then be fed into any of Gnuastro's
+programs directly, without having to decompress it separately (it will just
+take them a little longer, because they have to decompress it internally
+before use).
+@end cartouche
When the input is a 2D image, to inspect NoiseChisel's output you can
configure SAO DS9 in your Graphic User Interface (GUI) to open
@@ -14583,74 +15434,90 @@ function ds9-nc3d-check @{
@}
@end example
-@node MakeCatalog, Match, NoiseChisel, Data analysis
+
+
+
+
+
+
+
+
+
+@node MakeCatalog, Match, Segment, Data analysis
@section MakeCatalog
At the lowest level, a dataset (for example an image) is just a collection
of values, placed after each other in any number of dimensions (for example
an image is a 2D dataset). Each data-element (pixel) just has two
-properties: its position (relative to the rest) and its value. The entire
-input dataset (a large image for example) is rarely treated as a singular
-entity for higher-level analysis@footnote{In low-level
-reduction/preparation of a dataset, you do in fact treat a whole image as a
-single entity. You can derive the over-all properties of all the pixels in
-a dataset with Gnuastro's Statistics program (see @ref{Statistics})}. You
-want to know the properties of the scientifically interesting targets that
-are embedded in it. For example the magnitudes, positions and elliptical
-properties of the galaxies that are in the image. MakeCatalog is Gnuastro's
-program to derive higher-level information for @emph{pre-defined} regions
-of a dataset. The role of MakeCatalog in a scientific analysis and the
-benefits of this model of data-analysis (were detection/identification is
-separated from measurement) is discussed in
-@url{https://arxiv.org/abs/1611.06387v1, Akhlaghi [2016]}. We strongly
-recommend reading this short paper for a better understanding of this
-methodology and use MakeCatalog most effectively. However, that paper
-cannot undergo any more change, so this manual is the definitive guide.
-
-As discussed above, you have to define the regions of a dataset that you
-are interested in @emph{before} running MakeCatalog. MakeProfiles currently
-uses labeled dataset(s) for this job. A labeled dataset for a given input
-dataset has the size/dimensions as the input, but its pixels have an
-integer type (see @ref{Numeric data types})@footnote{If the program you
-used to generate the labeled image only outputs floating point types, but
-you know it only has integer valued pixels that are stored in a floating
-point container, you can use Gnuastro's Arithmetic program (see
-@ref{Arithmetic}) to change the numerical data type of the image
-(@file{flabel.fits}) to an integer type image (@file{label.fits}) with a
-command like below:@*@command{$ astarithmetic flabel.fits int32
---output=label.fits}}: all pixels with the same label (integers larger and
-equal to one) are used to generate the requested output columns of
-MakeCatalog for the row of their labeled value. For example, the flux
-weighted average position of all the pixels with a label of 42 will be
-considered as the central position@footnote{See @ref{Measuring elliptical
-parameters} for a discussion on this and the derivation of positional
-parameters.} of the 42nd row of the output catalog. Pixels with labels
-equal to or smaller than zero will be ignored by MakeCatalog. In other
-words, the number of rows of the output catalog will be determined from the
-labeled image.
-
-The labeled image maybe created with any tool@footnote{For example, you can
-even use a graphic user interface image editing tool like the GNU Image
-Manipulation Program (or GIMP) and use Gnuastro's ConvertType to convert it
-to a FITS file.}. Within Gnuastro you can use these two solutions depending
-on a-priori/parametric knowledge of the targets you want to study:
+properties: its position (relative to the rest) and its value. In
+higher-level analysis, an entire dataset (an image for example) is rarely
+treated as a singular entity@footnote{You can derive the over-all
+properties of a complete dataset (1D table column, 2D image, or 3D
+datacube) treated as a single entity with Gnuastro's Statistics program
+(see @ref{Statistics}).}. You usually want to know/measure the properties
+of the (separate) scientifically interesting targets that are embedded in
+it. For example the magnitudes, positions and elliptical properties of the
+galaxies that are in the image.
+
+MakeCatalog is Gnuastro's program for localized measurements over a
+dataset. The role of MakeCatalog in a scientific analysis and the benefits
+of this model of data analysis (were detection/segmentation is separated
+from measurement) is discussed in @url{https://arxiv.org/abs/1611.06387v1,
+Akhlaghi [2016]}. We strongly recommend reading this short paper for a
+better understanding of this methodology thus effective usage of
+MakeCatalog, in combination with the other Gnuastro's programs. However,
+that paper cannot undergo any more change, so this manual is the definitive
+guide.
+
+It is important to define your regions of interest @emph{before} running
+MakeCatalog. MakeCatalog is specialized in doing measurements accurately
+and efficiently. Therefore MakeCatalog will not do detection, segmentation,
+or defining apertures on requested positions in your dataset. Following
+Gnuastro's modularity principle, There are separate and higly specialized
+and customizable programs in Gnuastro for these other jobs:
+
@itemize
-@cindex Aperture photometry
-@cindex Photometry, aperture
@item
-When you already know the positions and parametric (for example circular or
-elliptical) properties of the targets, you can use @ref{MakeProfiles} to
-generate a labeled image from another catalog. This is also known as
-aperture photometry (the apertures are defined a-priori).
+@ref{Arithmetic}: Detection with a simple threshold.
+
+@item
+@ref{NoiseChisel}: Advanced detection.
@item
-When the shape of your targets cannot be parameterized accurately (for
-example galaxies), or if you don't know the number/shape of the targets in
-the image, you can use Gnuastro's NoiseChisel program to detect and segment
-(make labeled images of) the @emph{objects} and @emph{clumps} in the input
-image, see @ref{NoiseChisel}.
+@ref{Segment}: Segmentation (substructure over detections).
+
+@item
+@ref{MakeProfiles}: Aperture creation for known positions.
@end itemize
+These programs will/can return labeled dataset(s) to be fed into
+MakeCatalog. The labeled dataset must have the same size/dimensions as the
+input, but only with integer valued pixels that have the label/counter for
+the feature the pixel belongs to.
+
+These labels are then directly used to make the necessary measurements. For
+example the flux weighted average position of all the pixels with a label
+of 42 will be considered as the central position@footnote{See
+@ref{Measuring elliptical parameters} for a discussion on this and the
+derivation of positional parameters.} of the 42nd row of the output
+catalog. Similarly, the sum of all these pixels will be the 42nd row in the
+brightness column and etc. Pixels with labels equal to or smaller than zero
+will be ignored by MakeCatalog. In other words, the number of rows in
+MakeCatalog's output is already known before running it.
+
+Before getting into the details of running MakeCatalog (in @ref{Invoking
+astmkcatalog}, we'll start with a discussion on the basics of its approach
+to separating detection from measuremens in @ref{Detection and catalog
+production}. A very important factor in any measurement is understanding
+its validity range, or limits. Therefore in @ref{Quantifying measurement
+limits}, we'll discuss how to estimate the reliability of the detection and
+basic measurements. This section will continue with a derivation of
+elliptical parameters from the labelled datasets in @ref{Measuring
+elliptical parameters}. For those who feel MakeCatalog's existing
+measurements/columns aren't enough and would like to add further
+measurements, in @ref{Adding new columns to MakeCatalog}, a checklist of
+steps is provided for readily adding your own new measurements/columns.
+
@menu
* Detection and catalog production:: Discussing why/how to treat these
separately
* Quantifying measurement limits:: For comparing different catalogs.
@@ -14662,28 +15529,50 @@ image, see @ref{NoiseChisel}.
@node Detection and catalog production, Quantifying measurement limits,
MakeCatalog, MakeCatalog
@subsection Detection and catalog production
-As discussed above (@ref{MakeCatalog}), NoiseChisel (Gnuastro's signal
-detection tool, see @ref{NoiseChisel}) does not produce any catalog of the
-detected objects. However, most other common tools in astronomical
-data-analysis (for example
+Most other common tools in low-level astronomical data-analysis (for
+example
SExtractor@footnote{@url{https://www.astromatic.net/software/sextractor}})
-merge the two processes into one. Gnuastro's modularized methodology is
+merge the two processes of detection and measurement into one. Gnuastro's
+modularized methodology to separating detection from measurements is
therefore new to many experienced astronomers and deserves a short review
here. Further discussion on the benefits of this methodology can be seen in
@url{https://arxiv.org/abs/1611.06387v1, Akhlaghi [2016]}.
-To simplify catalog production from a raw input image in Gnuastro,
-NoiseChisel's output (see @ref{NoiseChisel output}) can be directly fed
-into MakeCatalog. This is good when no further customization is necessary
-and you want a fast/simple. But the modular approach taken by Gnuastro has
-many benefits that will become more apparent as you get more experienced in
-astronomical data analysis and want to be more creative in using your
-valuable data for the exciting scientific project you are working on. In
-short the reasons for this modularity can be classified as below:
+The raw output of any of the programs mentioned in @ref{MakeCatalog} can be
+directly fed into MakeCatalog to get to a fast catalog. This is good when
+no further customization is necessary and you want a fast/simple
+catalog. But the modular approach taken by Gnuastro has many benefits that
+will become clear as you get more experienced in astronomical data analysis
+and want to be more creative in using your valuable data for the exciting
+scientific project you are working on. In short the reasons for this
+modularity can be classified as below:
@itemize
@item
+Simplicity/robustness of independent, modular tools: making a catalog is a
+logically separate process from labeling (detection, segmentation, or
+aperture production). A user might want to do certain operations on the
+labeled regions before creating a catalog for them. Another user might want
+the properties of the same pixels/objects in another image (another filter
+for example) to measure the colors or SED fittings.
+
+Here is an example of doing both: suppose you have images in various
+broad band filters at various resolutions and orientations. The image
+of one color will thus not lie exactly on another or even be in the
+same scale. However, it is imperative that the same pixels be used in
+measuring the colors of galaxies.
+
+To solve the problem, NoiseChisel can be run on the reference image to
+generate the labeled detection image. After wards, the labeled image can be
+warped into the grid of the other color (using @ref{Warp}). MakeCatalog
+will then generate the same catalog for both colors (with the different
+labeled images). It is currently customary to warp the images to the same
+pixel grid, however, modification of the scientific dataset is very harmful
+for the data and creates correlated noise. It is much more accurate to do
+the transformations on the labeled image.
+
+@item
Complexity of a monolith: Adding in a catalog functionality to the detector
program will add several more steps (and many more options) to its
processing that can equally well be done outside of it. This makes
@@ -14694,32 +15583,11 @@ As an example, if the parameter you want to measure
over one profile is not
provided by the developers of MakeCatalog. You can simply open this tiny
little program and add your desired calculation easily. This process is
discussed in @ref{Adding new columns to MakeCatalog}. However, if making a
-catalog was part of NoiseChisel, it would require a lot of energy to
-understand all the steps and internal structures of that large program (the
-most complex in Gnuastro) in order to add desired parameter in a catalog.
-
-@item
-Simplicity/robustness of independent, modular tools: making a catalog is a
-logically separate process from labeling (detection and segmentation). A
-user might want to do certain operations on the labeled regions before
-creating a catalog for them. Another user might want the properties of the
-same pixels/objects in another image (another filter for example) to
-measure the colors or SED fittings.
-
-Here is an example of doing both: suppose you have images in various
-broad band filters at various resolutions and orientations. The image
-of one color will thus not lie exactly on another or even be in the
-same scale. However, it is imperative that the same pixels be used in
-measuring the colors of galaxies.
-
-To solve the problem, NoiseChisel can be run on the reference image to
-generate the labeled image. After wards, the labeled image can be warped
-into the grid of the other color (using @ref{Warp}). MakeCatalog will then
-generate the same catalog for both colors (with the different labeled
-images). It is currently customary to warp the images to the same pixel
-grid, however, modification of the scientific dataset is very harmful for
-the data and creates correlated noise. It is much more accurate to do the
-transformations on the labeled image.
+catalog was part of NoiseChisel for example, adding a new
+column/measurement would require a lot of energy to understand all the
+steps and internal structures of that huge program. It might even be so
+intertwined with its processing, that adding new columns might cause
+problems/bugs in its primary job (detection).
@end itemize
@@ -14961,10 +15829,10 @@ today, we can make customized segmentation maps for
each object.
If requested, MakeCatalog will estimate teh the upper limit magnitude is
found for each object in the image separately, the procedure is fully
-configurable with the options in @ref{Upper-limit magnitude settings}. If
-one value for the whole image is required, you can either use the surface
-brightness limit above or make a circular aperture and feed it into
-MakeCatalog to request an upper-limit magnitude for it.
+configurable with the options in @ref{Upper-limit settings}. If one value
+for the whole image is required, you can either use the surface brightness
+limit above or make a circular aperture and feed it into MakeCatalog to
+request an upper-limit magnitude for it.
@end table
@@ -15186,6 +16054,14 @@ column requires new raw calculations, add a row to the
respective list. If
your calculation requires any other settings parameters, you should add a
variable to the @code{mkcatalogparams} structure.
+@item ui.c
+If the new column needs raw calculations (an entry was added in
+@code{objectcols} and @code{clumpcols}), specify which inputs it needs in
+@code{ui_necessary_inputs}, similar to the other options. Afterwards, if
+your column includes any particular settings (you needed to add a variable
+to the @code{mkcatalogparams} structure in @file{main.h}), you should do
+the sanity checks and preparations for it here.
+
@item ui.h
The @code{option_keys_enum} associates a unique value for each option to
MakeCatalog. The options that have a short option version, the single
@@ -15193,6 +16069,7 @@ character short comment is used for the value. Those
that don't have a
short option version, get a large integer automatically. You should add a
variable here to identify your desired column.
+
@cindex GNU C library
@item args.h
This file specifies all the parameters for the GNU C library, Argp
@@ -15201,11 +16078,6 @@ new column, just copy an existing set of parameters
and change the first,
second and 5th values (the only ones that differ between all the columns),
you should use the macro you defined in @file{ui.h} here.
-@item ui.c
-If your column includes any particular settings (you added a variable to
-the @code{mkcatalogparams} structure in @file{main.h}), you should do the
-sanity checks and preparations for it here. Otherwise, you can ignore this
-file.
@item columns.c
This file contains the main definition and high-level calculation of your
@@ -15240,9 +16112,9 @@ Update this manual and add a description for the new
column.
@node Invoking astmkcatalog, , Adding new columns to MakeCatalog, MakeCatalog
@subsection Invoking MakeCatalog
-MakeCatalog will make a catalog from an input image and at least on labeled
-image. The executable name is @file{astmkcatalog} with the following
-general template
+MakeCatalog will do measurements and produce a catalog from a labeled
+dataset and optional values dataset(s). The executable name is
+@file{astmkcatalog} with the following general template
@example
$ astmkcatalog [OPTION ...] InputImage.fits
@@ -15253,216 +16125,223 @@ One line examples:
@example
## Create catalog with RA, Dec, Magnitude and Magnitude error,
-## `input.fits' is NoiseChisel's output:
-$ astmkcatalog --ra --dec --magnitude --magnitudeerr input.fits
+## from Segment's output:
+$ astmkcatalog --ra --dec --magnitude --magnitudeerr seg-out.fits
## Same catalog as above (using short options):
-$ asmkcatalog -rdmG input.fits
+$ asmkcatalog -rdmG seg-out.fits
-## Write the catalog to a FITS table:
-$ astmkcatalog -mpQ --output=cat.fits input_labeled.fits
+## Write the catalog to a text table:
+$ astmkcatalog -mpQ seg-out.fits --output=cat.txt
-## Read the columns to create from `columns.conf':
-$ astmkcatalog --config=columns.conf input_labeled.fits
+## Output columns specified in `columns.conf':
+$ astmkcatalog --config=columns.conf seg-out.fits
-## Use different images for the objects and clumps inputs:
-$ astmkcatalog --objectsfile=K_labeled.fits --objectshdu=1 \
- --clumpsfile=K_labeled.fits --clumpshdu=2 i_band.fits
+## Use object and clump labels from a K-band image, but pixel values
+## from an i-band image.
+$ astmkcatalog K_segmented.fits --hdu=DETECTIONS --clumpscat \
+ --clumpsfile=K_segmented.fits --clumpshdu=CLUMPS \
+ --valuesfile=i_band.fits
@end example
@cindex Gaussian
@noindent
-If MakeCatalog is to do processing, an input image should be provided with
-the recognized extensions as input data, see @ref{Arguments}. The options
-described in this section are those that are only particular to
-MakeProfiles. For operations that MakeProfiles shares with other programs
-(mainly involving input/output or general processing steps), see
-@ref{Common options}. Also see @ref{Common program behavior} for some
-general characteristics of all Gnuastro programs including MakeCatalog.
-
-MakeCatalog needs 4 (or 5) images as input. These images can be separate
-extensions in one file (NoiseChisel's default output), or each can have its
-own file and its own extension. See @ref{NoiseChisel output} for the
-list. The clump labels image is not mandatory (when no clump catalog is
-required, for example in aperture photometry). When inspecting the object
-labels image, MakeProfiles will look for a @code{WCLUMPS} (short for
-with-clumps) header keyword. If that keyword is present and has a value of
-@code{yes}, @code{1}, or @code{y} (case insensitive) then a clump image
-must also be provided and a clump catalog will be made. When @code{WCLUMPS}
-isn't present or has any other value, only an object catalog will be
-created and all clump related options/columns will be ignored.
-
-@cindex Photometry, aperture
-@cindex Aperture photometry
-For example, if you only need an object catalog from NoiseChisel's output,
-you can use Gnuastro's Fits program (see @ref{Fits}) to modify or remove
-the @code{WCLUMPS} keyword in the objects HDU, then run MakeCatalog on
-it. Another example can be aperture photometry: let's assume you have made
-your labeled image (defining the apertures) with MakeProfiles. Clumps are
-not defined in this context, so besides the input and labeled image, you
-only need NoiseChisel's Sky and Sky standard deviation images (run
-NoiseChisel with the @option{--onlydetection} option). Since MakeProfile's
-output doesn't contain the @code{WCLUMPS} keyword, you just have to specify
-your labeled image with the @option{--objectsfile} option and also set its
-HDU. Note that labeled images have to be an integer type. Therefore, if you
-are using MakeProfiles to define the apertures/labels, you can use its
-@option{--type=int32} for example, see @ref{Input output options} and
-@ref{Numeric data types}.
-
-When a clump catalog is also desired, two catalogs will be made: one for
-the objects (suffixed with @file{_o.txt} or @file{_o.fits}) and another for
-the clumps (suffixed with @file{_c.txt} or @file{_c.fits}). Therefore if
-any value is given to the @option{--output} option, MakeCatalogs will
-replace these two suffixes with any existing suffix in the given value. If
-no output value is given, MakeCatalog will use the input name, see
-@ref{Automatic output}. The format of the output table is specified with
-the @option{--tableformat} option, see @ref{Input output options}.
-
-When MakeCatalog is run on multiple threads, the clumps catalog rows will
-not be sorted by object since each object is processed independently by one
-thread and threaded applications are asynchronous. The clumps in each
-object will be sorted based on their labels, but you will find lower-index
-objects come after higher-index ones (especially if they have more clumps
-and thus take more time). If the order is very important for you, you can
-run the following command to sort the rows by object ID (and clump ID with
-each object):
-
-@example
-$ awk '!/^#/' out_c.txt | sort -g -k1,1 -k2,2
-@end example
+If MakeCatalog is to do processing (not printing help or option values), an
+input labeled image should be provided. The options described in this
+section are those that are particular to MakeProfiles. For operations that
+MakeProfiles shares with other programs (mainly involving input/output or
+general processing steps), see @ref{Common options}. Also see @ref{Common
+program behavior} for some general characteristics of all Gnuastro programs
+including MakeCatalog.
+
+The various measurements/columns of MakeCatalog are requested as options,
+either on the command-line or in configuration files, see
+@ref{Configuration files}. The full list of available columns is available
+in @ref{MakeCatalog output columns}. Depending on the requested columns,
+MakeCatalog needs more than one input dataset, for more details, please see
+@ref{MakeCatalog inputs and basic settings}. The upper-limit measurements
+in particular need several configuration options which are thoroughly
+discussed in @ref{Upper-limit settings}. Finally, in @ref{MakeCatalog
+output file} the output file(s) created by MakeCatalog are discussed.
@menu
-* MakeCatalog input files:: Specifying the different input files.
-* MakeCatalog general settings:: Options for general column settings.
-* Upper-limit magnitude settings:: Necessary to define upper-limit magnitudes.
-* MakeCatalog output columns:: How to specify the columns in the output.
+* MakeCatalog inputs and basic settings:: Input files and basic settings.
+* Upper-limit settings:: Settings for upper-limit measurements.
+* MakeCatalog output columns:: Available columns in MakeCatalog's output
table.
+* MakeCatalog output file:: File names of MakeCatalog's output table.
@end menu
-@node MakeCatalog input files, MakeCatalog general settings, Invoking
astmkcatalog, Invoking astmkcatalog
-@subsubsection MakeCatalog input files
+@node MakeCatalog inputs and basic settings, Upper-limit settings, Invoking
astmkcatalog, Invoking astmkcatalog
+@subsubsection MakeCatalog inputs and basic settings
+
+MakeCatalog works by using a localized/labeled dataset (see
+@ref{MakeCatalog}). This dataset maps/labels pixels to a specific target
+(row number in the final catalog) and is thus the only necessary input
+dataset to produce a minimal catalog in any situation. Because it only has
+labels/counters, it must have an integer type (see @ref{Numeric data
+types}), see below if your labels are in a floating point container. When
+the requested measurements only need this dataset (for example
+@option{--geox}, @option{--geoy}, or @option{--geoarea}), MakeCatalog won't
+read any more datasets.
+
+Low-level measurements that only use the labeled image are rarely
+sufficient for any high-level science case. Therefore necessary input
+datasets depend on the requested columns in each run. For example, let's
+assume you want the brightness/magnitude and signal-to-noise ratio of your
+labeled regions. For these columns, you will also need to provide an extra
+dataset containing values for every pixel of the labeled input (to measure
+brightness) and another for the Sky standard deviation (to measure
+error). All such auxiliary input files have to have the same size (number
+of pixels in each dimension) as the input labeled image. Their numeric data
+type is irrelevant (they will be converted to 32-bit floating point
+internally). For the full list of available measurements, see
+@ref{MakeCatalog output columns}.
+
+The ``values'' dataset is used for measurements like brightness/magnitude,
+or flux-weighted positions. If it is a real image, by default it is assumed
+to be already Sky-subtracted prior to running MakeCatalog. If it isn't, you
+use the @option{--subtractsky} option to, so MakeCatalog reads and
+subtracts the Sky dataset before any processing. To obtain the Sky value,
+you can use the @option{--sky} option of @ref{Statistics}, but the best
+recommended method is @ref{NoiseChisel}, see @ref{Sky value}.
+
+MakeCatalog can also do measurements on sub-structures of detections. In
+other words, it can produce two catalogs. Following the nomenclature of
+Segment (see @ref{Segment}), the main labeled input dataset is known as
+``object'' labels and the (optional) sub-structure input dataset is known
+as ``clumps''. If MakeCatalog is run with the @option{--clumpscat} option,
+it will also need a labeled image containing clumps, similar to what
+Segment produces (see @ref{Segment output}). Since clumps are defined
+within detected regions (they exist over signal, not noise), MakeCatalog
+uses their boundaries to subtract the level of signal under them.
+
+There are separate options to explicitly request a file name and
+HDU/extension for each of the required input datasets as fully described
+below (with the @option{--*file} format). When each dataset is in a
+separate file, these options are necessary. However, one great advantage of
+the FITS file format, that is heavily used in astronomy, is that it allows
+the storage of multiple datasets in one file. So in most situations (for
+example if you are using the outputs of @ref{NoiseChisel} or
+@ref{Segment}), all the necessary input datasets can be in one file.
+
+When none of the @option{--*file} options are given, MakeCatalog will
+assume the necessary input datasets are in the file given as its argument
+(without any option). When the Sky or Sky standard deviation datasets are
+necessary and the only @option{--*file} option called is
+@option{--valuesfile}, MakeCatalog will search for these datasets (with the
+default/given HDUs) in the file given to @option{--valuesfile} (before
+looking into the the main argument file).
+
+It may happen that your labeled objects image was created with a program
+that only outputs floating point files. However, you know it only has
+integer valued pixels that are stored in a floating point container. In
+such cases, you can use Gnuastro's Arithmetic program (see
+@ref{Arithmetic}) to change the numerical data type of the image
+(@file{float.fits}) to an integer type image (@file{int.fits}) with a
+command like below:
+
+@example
+@command{$ astarithmetic float.fits int32 --output=int.fits}
+@end example
-MakeCatalog needs multiple images as input: a values image, one (or two)
-labeled images and Sky and Sky standard deviation images. The options
-described in this section allow you to identify them. If you use the
-default output of NoiseChisel (see @ref{NoiseChisel output}) you don't have
-to worry about any of these options and just give NoiseChisel's output file
-to MakeCatalog as described in @ref{Invoking astmkcatalog}.
+To summarize: if the input file to MakeCatalog is the default/full output
+of Segment (see @ref{Segment output}) you don't have to worry about any of
+the @option{--*file} options below. You can just give Segment's output file
+to MakeCatalog as described in @ref{Invoking astmkcatalog}. To feed
+NoiseChisel's output into MakeCatalog, just change the labeled dataset's
+header (with @option{--hdu=DETECTIONS}). The full list of input dataset
+options and general setting options are described below.
@table @option
-@item -O STR
-@itemx --objectsfile=STR
-The file name of the object labels image, if the image is in another
-extension of the input file, calling this option is not mandatory, just
-specify the extension/HDU with the @option{--objectshdu} option.
-
-@item --objectshdu=STR
-The HDU/extension of the object labels image. Only pixels with values above
-zero will be considered. The objects label image has to be an integer data
-type (see @ref{Numeric data types}) and only pixels with a value larger
-than zero will be used. If this extension contains the @code{WCLUMPS}
-keyword with a value of @code{yes}, @code{1}, or @code{y} (not case
-sensitive), then MakeCatalog will also build a clumps catalog, see
-@ref{Invoking astmkcatalog}.
-
-@item -C STR
+@item -l STR
@itemx --clumpsfile=STR
-Similar to @option{--objlabs} but for the labels of the clumps. This is
-only necessary if the image containing clump labels is not in the input
-file and the objects image has a @code{WCLUMPS} keyword, see
-@option{--objectshdu}.
+The file containing the labeled clumps dataset when @option{--clumpscat} is
+called (see @ref{MakeCatalog output file}). When @option{--clumpscat} is
+called, but this option isn't, MakeCatalog will look into the main input
+file (given as an argument) for the required extension/HDU (value to
+@option{--clumpshdu}).
@item --clumpshdu=STR
-The HDU/extension of the object labels image. Only pixels with values above
-zero will be considered. The objects label image has to be an integer data
-type (see @ref{Numeric data types}) and only pixels with a value larger
-than zero will be used.
-
-@item -s STR
-@itemx --skyfile=STR
-File name of an image keeping the Sky value for each pixel.
+The HDU/extension of the clump labels dataset. Only pixels with values
+above zero will be considered. The clump lables dataset has to be an
+integer data type (see @ref{Numeric data types}) and only pixels with a
+value larger than zero will be used. See @ref{Segment output} for a
+description of the expected format.
+
+@item -v STR
+@itemx --valuesfile=STR
+The file name of the (sky-subtracted) values dataset. When any of the
+columns need values to associate with the input labels (for example to
+measure the brightness/magnitude of a galaxy), MakeCatalog will look into a
+``values'' for the respective pixel values. In most common processing, this
+is the actual astronomical image that the labels were defined, or detected,
+over. The HDU/extension of this dataset in the given file can be specified
+with @option{--valueshdu}. If this option is not called, MakeCatalog will
+look for the given extension in the main input file.
+
+@item --valueshdu=STR/INT
+The name or number (counting from zero) of the extension containing the
+``values'' dataset, see the descriptions above and those in
+@option{--valuesfile} for more.
+
+@item -s STR/FLT
+@itemx --insky=STR/FLT
+Sky value as a single number, or the file name containing a dataset
+(different values per pixel or tile). The Sky dataset is only necessary
+when @option{--subtractsky} is called or when a column directly related to
+the Sky value is requested (currently @option{--sky}).
+
+When the Sky dataset is necessary and this option is not called,
+MakeCatalog will assume it is a dataset first look into the
+@option{--valuesfile} (if it is given) and then the main input file (given
+as an argument). By default the values dataset is assumed to be already Sky
+subtracted, so this dataset is not necessary for many of the columns.
+
+This dataset may be tessellation, with one element per tile (see
+@option{--oneelempertile} of @ref{Processing options}).
@item --skyhdu=STR
-The HDU of the Sky value image.
-
-@item -t STR
-@itemx --stdfile=STR
-File name of image keeping the Sky value standard deviation for each pixel.
+HDU/extension of the Sky dataset, see @option{--skyfile}.
+
+@item --subtractsky
+Subtract the sky value or dataset from the values file prior to any
+processing.
+
+@item -t STR/FLT
+@itemx --instd=STR/FLT
+Sky standard deviation value as a single number, or the file name
+containing a dataset (different values per pixel or tile). With the
+@option{--variance} option you can tell MakeCatalog to interpret this
+value/dataset as a variance image, not standard deviation.
+
+@strong{Important note:} This must only be the SKY standard deviation or
+variance (not including the signal's contribution to the error). In other
+words, the final standard deviation of a pixel depends on how much signal
+there is in it. MakeCatalog will find the amount of signal within each
+pixel (while subtracting the Sky, if @option{--subtractsky} is called) and
+account for the extra error due to it's value (signal). Therefore if the
+input standard deviation (or variance) image also contains the contribution
+of signal to the error, then the final error measurements will be
+over-estimated.
@item --stdhdu=STR
The HDU of the Sky value standard deviation image.
-@end table
-
-
-
-@node MakeCatalog general settings, Upper-limit magnitude settings,
MakeCatalog input files, Invoking astmkcatalog
-@subsubsection MakeCatalog general settings
-
-Some of the columns require particular settings (for example the zero point
-magnitude for measuring magnitudes), the options in this section can be
-used for such configurations.
-
-@table @option
+@item --variance
+The dataset given to @option{--stdfile} (and @option{--stdhdu} has the Sky
+variance of every pixel, not the Sky standard deviation.
@item -z FLT
@itemx --zeropoint=FLT
The zero point magnitude for the input image, see @ref{Flux Brightness and
magnitude}.
-@item -E
-@itemx --skysubtracted
-If the image has already been sky subtracted by another program, then you
-need to notify MakeCatalog through this option. Note that this is only
-relevant when the Signal to noise ratio is to be calculated.
-
-@item -T FLT
-@itemx --threshold=FLT
-For all the columns, only consider pixels that are above a given relative
-threshold. Symbolizing the value of this option as @mymath{T}, the Sky for
-a pixel at @mymath{(i,j)} with @mymath{\mu_{ij}} and its Standard deviation
-with @mymath{\sigma_{ij}}, that pixel will only be used if its value
-(@mymath{B_{ij}}) satisfies this condition:
-@mymath{B_{ij}>\mu_{ij}+{T}\sigma_{ij}}. The only calculations that will
-not be affected are is the average river values (@option{--riverave}),
-since they are used as a reference. A commented row will be added in the
-header of the output catalog that will print the given value, since this is
-a very important issue, it starts with @command{**IMPORTANT**}.
-
-NoiseChisel will detect very diffuse signal which is useful in most
-cases where the aggregate properties of the detections are desired,
-since there is signal there (with the desired certainty). However, in
-some cases, only the properties of the peaks of the objects/clumps are
-desired, for example in attempting to separate stars from galaxies,
-the peaks are the major target and the diffuse regions only act to
-complicate the separation. With this option, MakeCatalog will simply
-ignore any pixel below the relative threshold.
-
-@cindex NaN
-This option is not mandatory, so if it isn't given (after reading the
-command-line and all configuration files, see @ref{Configuration
-files}), MakeCatalog will still operate. However, if it has a value in
-any lower-level configuration file and you want to ignore that value
-for this particular run or in a higher-level configuration file, then
-set it to NaN, for example @option{--threshold=nan}. Gnuastro uses the
-C library's @code{strtod} function to read floats, which is not
-case-sensitive in reading NaN values. But to be consistent, it is good
-practice to only use @option{nan}.
-
-@item --nsigmag=FLT
-The median standard deviation (from the standard deviation image) will be
-multiplied by the value to this option and its magnitude will be reported
-in the comments of the output catalog. This value is a per-pixel value, not
-per object/clump and is not found over an area or aperture, like the common
-@mymath{5\sigma} values that are commonly reported as a measure of depth or
-the upper-limit measurements (see @ref{Quantifying measurement
-limits}).
-
@end table
-@node Upper-limit magnitude settings, MakeCatalog output columns, MakeCatalog
general settings, Invoking astmkcatalog
-@subsubsection Upper-limit magnitude settings
+@node Upper-limit settings, MakeCatalog output columns, MakeCatalog inputs and
basic settings, Invoking astmkcatalog
+@subsubsection Upper-limit settings
The upper limit magnitude was discussed in @ref{Quantifying measurement
@@ -15568,10 +16447,27 @@ The multiple of the final (@mymath{\sigma}-clipped)
standard deviation (or
@mymath{\sigma}) used to measure the upper-limit brightness or
magnitude.
+@item --checkupperlimit=INT[,INT]
+Print a table of positions and measured values for all the full random
+distribution used for one particular object or clump. If only one integer
+is given to this option, it is interpretted to be an object's label. If two
+values are given, the first is the object label and the second is the ID of
+requested clump wihtin it.
+
+The output is a table with three columns (its type is determined with the
+@option{--tableformat} option, see @ref{Input output options}). The first
+two columns are the position of the first pixel in each random sampling of
+this particular object/clump. The the third column is the measured flux
+over that region. If the region overlapped with a detection or masked
+pixel, then its measured value will be a NaN (not-a-number). The total
+number of rows is thus un-known, but you can be sure that the number of
+rows with non-NaN measurements is the number given to the @option{--upnum}
+option.
+
@end table
-@node MakeCatalog output columns, , Upper-limit magnitude settings, Invoking
astmkcatalog
+@node MakeCatalog output columns, MakeCatalog output file, Upper-limit
settings, Invoking astmkcatalog
@subsubsection MakeCatalog output columns
The final group of options particular to MakeCatalog are those that specify
@@ -15604,22 +16500,6 @@ if only object catalogs are required, it has the same
effect as
@item --idinhostobj
[Clumps] The ID of this clump in its host object.
-@item -C
-@itemx --numclumps
-[Objects] The number of clumps in this object.
-
-@item -a
-@itemx --area
-The raw area (number of pixels) in any clump or object independent of what
-pixel it lies over (if it is NaN/blank or unused for example).
-
-@item --clumpsarea
-[Objects] The total area of all the clumps in this object.
-
-@item --weightarea
-The area (number of pixels) used in the flux weighted position
-calculations.
-
@item -x
@itemx --x
The flux weighted center of all objects and clumps along the first FITS
@@ -15765,6 +16645,10 @@ If no usable pixels (blank or below the threshold) are
present over the
clump or object, the stored value will be NaN (note that zero is
meaningful).
+@item --brightnesserr
+The (@mymath{1\sigma}) error in measuring the brightness of objects or
+clumps.
+
@item --clumpbrightness
[Objects] The total brightness of the clumps within an object. This is
simply the sum of the pixels associated with clumps in the object. If no
@@ -15772,16 +16656,6 @@ usable pixels (blank or below the threshold) are
present over the clump or
object, the stored value will be NaN, because zero (note that zero is
meaningful).
-@item --mean
-The mean sky subtracted value of pixels within the object or clump. For
-clumps, the average river flux is subtracted from the sky subtracted
-mean.
-
-@item --median
-The median sky subtracted value of pixels within the object or clump. For
-clumps, the average river flux is subtracted from the sky subtracted
-median.
-
@item --noriverbrightness
[Clumps] The Sky (not river) subtracted clump brightness. By definition,
for the clumps, the average brightness of the rivers surrounding it are
@@ -15805,6 +16679,16 @@ If no usable pixels (blank or below the possibly given
threshold) are
present over the clump or object, the stored value will be NaN (note that
zero is meaningful).
+@item --mean
+The mean sky subtracted value of pixels within the object or clump. For
+clumps, the average river flux is subtracted from the sky subtracted
+mean.
+
+@item --median
+The median sky subtracted value of pixels within the object or clump. For
+clumps, the average river flux is subtracted from the sky subtracted
+median.
+
@item -m
@itemx --magnitude
The magnitude of clumps or objects, see @option{--brightness}.
@@ -15828,20 +16712,20 @@ defined for work on adding these sources of error too.
@item --upperlimit
The upper limit value (in units of the input image) for this object or
clump. See @ref{Quantifying measurement limits} and @ref{Upper-limit
-magnitude settings} for a complete explanation. This is very important for
-the fainter and smaller objects in the image where the measured magnitudes
-are not reliable.
+settings} for a complete explanation. This is very important for the
+fainter and smaller objects in the image where the measured magnitudes are
+not reliable.
@item --upperlimitmag
The upper limit magnitude for this object or clump. See @ref{Quantifying
-measurement limits} and @ref{Upper-limit magnitude settings} for a complete
+measurement limits} and @ref{Upper-limit settings} for a complete
explanation. This is very important for the fainter and smaller objects in
the image where the measured magnitudes are not reliable.
@item --upperlimitonesigma
The @mymath{1\sigma} upper limit value (in units of the input image) for
this object or clump. See @ref{Quantifying measurement limits} and
-@ref{Upper-limit magnitude settings} for a complete explanation. When
+@ref{Upper-limit settings} for a complete explanation. When
@option{--upnsigma=1}, this column's values will be the same as
@option{--upperlimit}.
@@ -15849,16 +16733,30 @@ this object or clump. See @ref{Quantifying
measurement limits} and
The position of the total brightness measured within the distribution of
randomly placed upperlimit measurements in units of the distribution's
@mymath{\sigma} or standard deviation. See @ref{Quantifying measurement
-limits} and @ref{Upper-limit magnitude settings} for a complete
-explanation.
+limits} and @ref{Upper-limit settings} for a complete explanation.
@item --upperlimitquantile
The position of the total brightness measured within the distribution of
randomly placed upperlimit measurements as a quantile (value between 0 or
-1). See @ref{Quantifying measurement limits} and @ref{Upper-limit magnitude
-settings} for a complete explanation. If the object is brighter than the
-brightest randomly placed profile, a value of @code{inf} is returned. If it
-is less than the minimum, a value of @code{-inf} is reported.
+1). See @ref{Quantifying measurement limits} and @ref{Upper-limit settings}
+for a complete explanation. If the object is brighter than the brightest
+randomly placed profile, a value of @code{inf} is returned. If it is less
+than the minimum, a value of @code{-inf} is reported.
+
+@item --upperlimitskew
+@cindex Skewness
+This column contains the nonparametric skew of the sigma-clipped random
+distribution that was used to estimate the upper-limit magnitude. Taking
+@mymath{\mu} as the mean, @mymath{\nu} as the median and @mymath{\sigma} as
+the standard deviation, the traditional definition of skewness is defined
+as: @mymath{(\mu-\nu)/\sigma}.
+
+This can be a good measure to see how much you can trust the random
+measurements, or in other words, how accurately the regions with signal
+have been masked/detected. If the skewness is strong (and to the positive),
+then you can tell that you have a lot of un-detected signal in the dataset,
+and therefore that the upper-limit measurement (and other measurements) are
+not reliable.
@item --riverave
[Clumps] The average brightness of the river pixels around this
@@ -15890,6 +16788,28 @@ The sky value standard deviation (per pixel) for this
clump or
object. Like @option{--sky}, this is the average of the values in the
input sky standard deviation image pixels that lie over this object.
+@item -C
+@itemx --numclumps
+[Objects] The number of clumps in this object.
+
+@item -a
+@itemx --area
+The raw area (number of pixels) in any clump or object independent of what
+pixel it lies over (if it is NaN/blank or unused for example).
+
+@item --clumpsarea
+[Objects] The total area of all the clumps in this object.
+
+@item --weightarea
+The area (number of pixels) used in the flux weighted position
+calculations.
+
+@item --geoarea
+The area of all the pixels labeled with an object or clump. Note that
+unlike @option{--area}, pixel values are completely ignored in this
+column. For example, if a pixel value is blank, it won't be counted in
+@option{--area}, but will be counted here.
+
@item -A
@itemx --semimajor
The pixel-value weighted semi-major axis of the profile (assuming it is an
@@ -15900,6 +16820,10 @@ ellipse) in units of pixels. See @ref{Measuring
elliptical parameters}.
The pixel-value weighted semi-minor axis of the profile (assuming it is an
ellipse) in units of pixels. See @ref{Measuring elliptical parameters}.
+@item --axisratio
+The pixel-value weighted axis ratio (semi-minor/semi-major) of the object
+or clump.
+
@item -p
@itemx --positionangle
The pixel-value weighted angle of the semi-major axis with the first FITS
@@ -15913,6 +16837,10 @@ assuming it is an ellipse.
The geometric (ignoring pixel values) semi-minor axis of the profile,
assuming it is an ellipse.
+@item --geoaxisratio
+The geometric (ignoring pixel values) axis ratio of the profile, assuming
+it is an ellipse.
+
@item --geopositionangle
The geometric (ignoring pixel values) angle of the semi-major axis
with the first FITS axis in degrees.
@@ -15922,6 +16850,92 @@ with the first FITS axis in degrees.
+@node MakeCatalog output file, , MakeCatalog output columns, Invoking
astmkcatalog
+@subsubsection MakeCatalog output file
+
+When complete, MakeCatalog will store its measurements as a table. If an
+output filename is given (see @option{--output} in @ref{Input output
+options}), the format of the table will be deduced from the name. When it
+isn't given, the input name will be appended with a @file{_cat} suffix (see
+@ref{Automatic output}) and its format will be determined from the
+@option{--tableformat} option, which is also discussed in @ref{Input output
+options}. @option{--tableformat} is also necessary when the requested
+output name is a FITS table (recall that FITS can accept ASCII and binary
+tables, see @ref{Table}).
+
+By default only a single catalog/table will be created for ``objects'',
+however, if @option{--clumpscat} is called, a secondary catalog/table will
+also be created. For more on ``objects'' and ``clumps'', see
+@ref{Segment}. In short, if you only have one set of labeled images, you
+don't have to worry about clumps (they are deactivated by default). Here is
+a full description of MakeCatalog's output options:
+
+@table @option
+@item -C
+@itemx --clumpscat
+Do measurements on clumps and produce a second catalog (only devoted to
+clumps). When this option is given, MakeCatalog will also look for a
+secondary labeled dataset (identifying substructure) and produce a catalog
+from that. For more on the definition on ``clumps'', see @ref{Segment}.
+
+When the output is a FITS file, the objects and clumps catalogs/tables will
+be stored as multiple extensions of one FITS file. You can use @ref{Table}
+to inspect the column meta-data and contents in this case. However, in
+plain text format (see @ref{Gnuastro text table format}), it is only
+possible to keep one table per file. Therefore, if the output is a text
+file, two ouput files will be created, ending in @file{_o.txt} (for
+objects) and @file{_c.txt} (for clumps).
+
+@item --noclumpsort
+Don't sort the clumps catalog based on object ID (only relevant with
+@option{--clumpscat}). This option will benefit the
+performance@footnote{The performance boost due to @option{--noclumpsort}
+can only be felt when there are a huge number of objects. Therefore, by
+default the output is sorted to avoid miss-understandings or bugs in the
+user's scripts when the user forgets to sort the outputs.} of MakeCatalog
+when it is run on multiple threads @emph{and} the position of the rows in
+the clumps catalog is irrelevant (for example you just want the
+number-counts).
+
+MakeCatalog does all its measurements on each @emph{object} independently
+and in parallel. As a result, while it is writing the measurements on each
+object's clumps, it doesn't know how many clumps there were in previous
+objects. Each thread will just fetch the first available row and write the
+information of clumps (in order) starting from that row. After all the
+measurements are done, by default (when this option isn't called),
+MakeCatalog will reorder/permute the clumps catalog to have both the object
+and clump ID in an ascending order.
+
+If you would like to order the catalog later (when its a plain text file),
+you can run the following command to sort the rows by object ID (and clump
+ID within each object), assuming they are respectively the first and second
+columns:
+
+@example
+$ awk '!/^#/' out_c.txt | sort -g -k1,1 -k2,2
+@end example
+
+@item --sfmagnsigma=FLT
+The median standard deviation (from a @command{MEDSTD} keyword in the Sky
+standard deviation image) will be multiplied by the value to this option
+and its magnitude will be reported in the comments of the output
+catalog. This value is a per-pixel value, not per object/clump and is not
+found over an area or aperture, like the common @mymath{5\sigma} values
+that are commonly reported as a measure of depth or the upper-limit
+measurements (see @ref{Quantifying measurement limits}).
+
+@item --sfmagarea=FLT
+Area (in arcseconds squared) to convert the per-pixel estimation of
+@option{--sfmagnsigma} in the comments section of the output tables. Note
+that this is just a unit conversion using the World Coordinate System (WCS)
+information in the input's header. It does not actually do any measurements
+on this area. For random measurements on any area, please use the
+upper-limit columns of MakeCatalog (see the discussion on upper-limit
+measurements in @ref{Quantifying measurement limits}).
+@end table
+
+
+
@node Match, , MakeCatalog, Data analysis
@section Match
@@ -17035,10 +18049,10 @@ Point source with `@code{point}' or `@code{4}'.
Flat profile with `@code{flat}' or `@code{5}'.
@item
-Circumference profile with `@code{circum}' or `@code{6}'. A fixed
-value will be used for all pixels between the truncation radius
-(@mymath{r_t}) and @mymath{r_t-w} (@mymath{w} is the value to the
-@option{--circumwidth}).
+Circumference profile with `@code{circum}' or `@code{6}'. A fixed value
+will be used for all pixels less than or equal to the truncation radius
+(@mymath{r_t}) and greater than @mymath{r_t-w} (@mymath{w} is the value to
+the @option{--circumwidth}).
@item
@item
@@ -17733,12 +18747,12 @@ distribution with
@cindex ADU
@cindex Gain
@cindex Counts
-This type of noise is completely independent of the type of objects
-being studied, it is completely determined by the instrument. So the
-flux scale (and not magnitude scale) is most commonly used for this
-type of noise. In practice, this value is usually reported in ADUs not
-flux or electron counts. The gain value of the device can be used to
-convert between these two, see @ref{Flux Brightness and magnitude}.
+This type of noise is independent of the signal in the dataset, it is only
+determined by the instrument. So the flux scale (and not magnitude scale)
+is most commonly used for this type of noise. In practice, this value is
+usually reported in ADUs not flux or electron counts. The gain value of the
+device can be used to convert between these two, see @ref{Flux Brightness
+and magnitude}.
@node Final noised pixel value, Generating random numbers, Instrumental noise,
Noise basics
@subsubsection Final noised pixel value
@@ -19388,10 +20402,11 @@ documentation will correspond to your installed
version.
* Library data container:: General data container in Gnuastro.
* Dimensions:: Dealing with coordinates and dimensions.
* Linked lists:: Various types of linked lists.
+* Array input output:: Reading and writing images or cubes.
+* Table input output:: Reading and writing table columns.
* FITS files:: Working with FITS data.
+* File input output:: Reading and writing to various file formats.
* World Coordinate System:: Dealing with the world coordinate system.
-* Text files:: Functions to work on Text files.
-* Table input output:: Reading and writing table columns.
* Arithmetic on datasets:: Arithmetic operations on a dataset.
* Tessellation library:: Functions for working on tiles.
* Bounding box:: Finding the bounding box.
@@ -20085,6 +21100,17 @@ that corresponds to its type. If @code{input} is a
tile over a larger
dataset, only the region that the tile covers will be set to blank.
@end deftypefun
+@deftypefun int gal_blank_is (void @code{*pointer}, uint8_t @code{type})
+Return 1 if the contents of @code{pointer} (assuming a type of @code{type})
+is blank. Otherwise, return 0. Note that this function only works on one
+element of the given type. So if @code{pointer} is an array, only its first
+element will be checked. Therefore for strings, the type of @code{pointer}
+is assumed to be @code{char *}. To check if an array/dataset has blank
+elements or to find which elements in an array are blank, you can use
+@code{gal_blank_present} or @code{gal_blank_flag} respectively (described
+below).
+@end deftypefun
+
@deftypefun int gal_blank_present (gal_data_t @code{*input}, int
@code{updateflag})
Return 1 if the dataset has a blank value and zero if it doesn't. Before
@@ -20123,6 +21149,11 @@ adjust the size properly (the number of non-blank
elements). In practice
this function doesn't @code{realloc} the input array, it just shifts the
blank elements to the end and adjusts the size elements of the
@code{gal_data_t}, see @ref{Generic data container}.
+
+If all the elements were blank, then @code{input->size} will be zero. This
+is thus a good parameter to check after calling this function to see if
+there actually were any non-blank elements in the input or not and take the
+appropriate measure. This can help avoid strange bugs in later steps.
@end deftypefun
@deftypefun {char *} gal_blank_as_string (uint8_t @code{type}, int
@code{width})
@@ -20566,7 +21597,9 @@ The functions in this section describes Gnuastro's
facilities to copy a
given dataset into another. The new dataset can have a different type
(including a string), it can be already allocated (in which case only the
values will be written into it). In all these cases, if the input dataset
-is a tile, only the data within the tile are copied.
+is a tile or a list, only the data within the given tile, or the given node
+in a list, are copied. If the input is a list, the @code{next} pointer will
+also be copied to the output, see @ref{List of gal_data_t}.
In many of the functions here, it is possible to copy the dataset to a new
numeric data type (see @ref{Numeric data types}. In such cases, Gnuastro's
@@ -20576,22 +21609,28 @@ fit into the output type.
@deftypefun {gal_data_t *} gal_data_copy (gal_data_t @code{*in})
Return a new dataset that is a copy of @code{in}, all of @code{in}'s
-meta-data will also copied into the output, except for @code{block}.
+meta-data will also copied into the output, except for @code{block}. If the
+dataset is a tile/list, only the given tile/node will be copied, the
+@code{next} pointer will also be copied however.
@end deftypefun
@deftypefun {gal_data_t *} gal_data_copy_to_new_type (gal_data_t @code{*in},
uint8_t @code{newtype})
Return a copy of the dataset @code{in}, converted to @code{newtype}, see
@ref{Library data types} for Gnuastro library's type identifiers. The
returned dataset will have all meta-data except their type and @code{block}
-equal to the input's metadata.
+equal to the input's metadata. If the dataset is a tile/list, only the
+given tile/node will be copied, the @code{next} pointer will also be copied
+however.
@end deftypefun
@deftypefun {gal_data_t *} gal_data_copy_to_new_type_free (gal_data_t
@code{*in}, uint8_t @code{newtype})
Return a copy of the dataset @code{in} that is converted to @code{newtype}
and free the input dataset. See @ref{Library data types} for Gnuastro
library's type identifiers. The returned dataset will have all meta-data,
-except their type, equal to the input's metadata. This function is similar
-to @code{gal_data_copy_to_new_type}, except that it will free the input
+except their type, equal to the input's metadata (including
+@code{next}). Note that if the input is a tile within a larger block, it
+will not be freed. This function is similar to
+@code{gal_data_copy_to_new_type}, except that it will free the input
dataset.
@end deftypefun
@@ -20749,7 +21788,7 @@ This macro works fully within its own @code{@{@}} block
and except for the
within this macro's block start with @code{gdn_}.
@end deffn
-@node Linked lists, FITS files, Dimensions, Gnuastro library
+@node Linked lists, Array input output, Dimensions, Gnuastro library
@subsection Linked lists (@file{list.h})
@cindex Array
@@ -21508,9 +22547,9 @@ In this example multiple images are linked together as
a list:
@example
size_t minmapsize=-1;
gal_data_t *tmp, *list=NULL;
-tmp = gal_fits_img_read("file1.fits", "1", minmapsize, 0, 0);
+tmp = gal_fits_img_read("file1.fits", "1", minmapsize);
gal_list_data_add( &list, tmp );
-tmp = gal_fits_img_read("file2.fits", "1", minmapsize, 0, 0);
+tmp = gal_fits_img_read("file2.fits", "1", minmapsize);
gal_list_data_add( &list, tmp );
@end example
@end deftypefun
@@ -21540,139 +22579,388 @@ Free all the datasets in @code{list} along with all
the allocated spaces in
each.
@end deftypefun
-@node FITS files, World Coordinate System, Linked lists, Gnuastro library
-@subsection FITS files (@file{fits.h})
-
-@cindex FITS
-@cindex CFITSIO
-The FITS format is the most common format to store data (images and tables)
-in astronomy. The CFITSIO library already provides a very good low-level
-collection of functions for manipulating FITS data. The low-level nature of
-CFITSIO is defined for versatility and portability. As a result, even a
-simple and basic operation, like reading an image or table column into
-memory, will require a special sequence of CFITSIO function calls which can
-be inconvenient and buggy to manage in separate locations. To ease this
-process, Gnuastro's library provides wrappers for CFITSIO functions. With
-these, it much easier to read, write, or modify FITS file data, header
-keywords and extensions. Hence, if you feel these functions don't exactly
-do what you want, we strongly recommend reading the CFITSIO manual to use
-its great features directly (afterwards, send us your wrappers so we can
-include it here for others to benefit also).
-All the functions and macros introduced in this section are declared in
-@file{gnuastro/fits.h}. When you include this header, you are also
-including CFITSIO's @file{fitsio.h} header. So you don't need to explicitly
-include @file{fitsio.h} anymore and can freely use any of its macros or
-functions in your code along with those discussed here.
-@menu
-* FITS macros errors filenames:: General macros, errors and checking names.
-* CFITSIO and Gnuastro types:: Conversion between FITS and Gnuastro types.
-* FITS HDUs:: Opening and getting information about HDUs.
-* FITS header keywords:: Reading and writing FITS header keywords.
-* FITS arrays:: Reading and writing FITS images/arrays.
-* FITS tables:: Reading and writing FITS tables.
-@end menu
-@node FITS macros errors filenames, CFITSIO and Gnuastro types, FITS files,
FITS files
-@subsubsection FITS Macros, errors and filenames
+@node Array input output, Table input output, Linked lists, Gnuastro library
+@subsection Array input output
-Some general constructs provided by Gnuastro's FITS handling functions are
-discussed here. In particular there are several useful functions about FITS
-file names.
+Getting arrays (commonly images or cubes) from a file into your program or
+writing them after the processing into an output file are some of the most
+common operations. The functions in this section are designed for such
+operations with the known file types. The functions here are thus just
+wrappers around funcitons of lower-level file type functions of this
+library, for example @ref{FITS files} or @ref{TIFF files}. If the file type
+of the input/output file is already known, you can use the functions in
+those sections respectively.
-@deffn Macro GAL_FITS_MAX_NDIM
-The maximum number of dimensions a dataset can have in FITS format,
-according to the FITS standard this is 999.
-@end deffn
+@deftypefun int gal_array_name_recognized (char @code{*filename})
+Return 1 if the given file name corresponds to one of the recognized file
+types for reading arrays.
+@end deftypefun
-@deftypefun void gal_fits_io_error (int @code{status}, char @code{*message})
-If @code{status} is non-zero, this function will print the CFITSIO error
-message corresponding to status, print @code{message} (optional) in the
-next line and abort the program. If @code{message==NULL}, it will print a
-default string after the CFITSIO error.
+@deftypefun gal_data_t gal_array_read (char @code{*filename}, char
@code{*extension}, size_t @code{minmapsize})
+Read the array within the given extension (@code{extension}) of
+@code{filename}. If the array is larger than @code{minmapsize} bytes, then
+it won't be read into RAM, but a file on the HDD/SSD (no difference for the
+programmer). @code{extension} will be ignored for files that don't support
+them (for example JPEG or text). For FITS files, @code{extension} can be a
+number or a string (name of the extension), but for TIFF files, it has to
+be number. In both cases, counting starts from zero.
+
+For multi-channel formats (like RGB images in JPEG or TIFF), this function
+will return a @ref{List of gal_data_t}: one data structure per
+channel. Thus if you just want a single array (and want to check if the
+user hasn't given a multi-channel input), you can check the @code{next}
+pointer of the returned @code{gal_data_t}.
@end deftypefun
-@deftypefun int gal_fits_name_is_fits (char @code{*name})
-If the @code{name} is an acceptable CFITSIO FITS filename return @code{1}
-(one), otherwise return @code{0} (zero). The currently acceptable FITS
-suffixes are @file{.fits}, @file{.fit}, @file{.fits.gz}, @file{.fits.Z},
-@file{.imh}, @file{.fits.fz}. IMH is the IRAF format which is acceptable to
-CFITSIO.
+@deftypefun void gal_array_read_to_type (char @code{*filename}, char
@code{*extension}, uint8_t @code{type}, size_t @code{minmapsize})
+Similar to @code{gal_array_read}, but the output data structure(s) will
+have a numeric data type of @code{type}, see @ref{Numeric data types}.
@end deftypefun
-@deftypefun int gal_fits_suffix_is_fits (char @code{*suffix})
-Similar to @code{gal_fits_name_is_fits}, but only for the suffix. The
-suffix doesn't have to start with `@key{.}': this function will return
-@code{1} (one) for both @code{fits} and @code{.fits}.
+@deftypefun void gal_array_read_one_ch (char @code{*filename}, char
@code{*extension}, size_t @code{minmapsize})
+@cindex Channel
+@cindex Color channel
+Read the dataset within @code{filename} (extension/hdu/dir
+@code{extension}) and make sure it is only a single channel. Formats like
+JPEG or TIFF support multiple channels per input, but it may happen that
+your program only works on a single dataset. This function can be a
+convenient way to make sure that the data that comes into your program is
+only one channel. It is just a very simple wrapper around
+@code{gal_array_read} that checks if there was more than one dataset and
+aborts with an informative error if there is more than one channel in the
+dataset.
@end deftypefun
-@deftypefun {char *} gal_fits_name_save_as_string (char @code{*filename}, char
@code{*hdu})
-If the name is a FITS name, then put a @code{(hdu: ...)} after it and
-return the string. If it isn't a FITS file, just print the name. Note that
-the space is allocated. This function is useful when you want to report a
-random file to the user which may be FITS or not (for a FITS file, simply
-the filename is not enough, the HDU is also necessary).
+@deftypefun void gal_array_read_one_ch_to_type (char @code{*filename}, char
@code{*extension}, uint8_t @code{type}, size_t @code{minmapsize})
+Similar to @code{gal_array_read_one_ch}, but the output data structure will
+has a numeric data type of @code{type}, see @ref{Numeric data types}.
@end deftypefun
+@node Table input output, FITS files, Array input output, Gnuastro library
+@subsection Table input output (@file{table.h})
-@node CFITSIO and Gnuastro types, FITS HDUs, FITS macros errors filenames,
FITS files
-@subsubsection CFITSIO and Gnuastro types
+Tables are a collection of one dimensional datasets that are packed
+together into one file. They are the single most common format to store
+high-level (processed) information, hence they play a very important role
+in Gnuastro. For a more thorough introduction, please see
+@ref{Table}. Gnuastro's Table program, and all the other programs that can
+read from and write into tables, use the functions of this section for
+reading and writing their input/output tables. For a simple demonstration
+of using the constructs introduced here, see @ref{Library demo - reading
+and writing table columns}.
-Both Gnuastro and CFITSIO have special identifiers for each type that they
-accept. Gnuastro's type identifiers are fully described in @ref{Library
-data types} and are usable for all kinds of datasets (images, table columns
-and etc) as part of Gnuastro's @ref{Generic data container}. However,
-following the FITS standard, CFITSIO has different identifiers for images
-and tables. Following CFITSIO's own convention, we will use @code{bitpix}
-for image type identifiers and @code{datatype} for its internal identifiers
-(and mainly used in tables). The functions introduced in this section can
-be used to convert between CFITSIO and Gnuastro's type identifiers.
+Currently only plain text (see @ref{Gnuastro text table format}) and FITS
+(ASCII and binary) tables are supported by Gnuastro. However, the low-level
+table infra-structure is written such that accommodating other formats is
+also possible and in future releases more formats will hopefully be
+supported. Please don't hesitate to suggest your favorite format so it can
+be implemented when possible.
-One important issue to consider is that CFITSIO's types are not fixed width
-(for example @code{long} may be 32-bits or 64-bits on different
-systems). However, Gnuastro's types are defined by their width. These
-functions will use information on the host system to do the proper
-conversion, so it strongly recommended to use these functions for
-portability of your code and not to assume a fixed correspondence between
-CFITSIO and Gnuastro's types.
+@deffn Macro GAL_TABLE_DEF_WIDTH_STR
+@deffnx Macro GAL_TABLE_DEF_WIDTH_INT
+@deffnx Macro GAL_TABLE_DEF_WIDTH_LINT
+@deffnx Macro GAL_TABLE_DEF_WIDTH_FLT
+@deffnx Macro GAL_TABLE_DEF_WIDTH_DBL
+@deffnx Macro GAL_TABLE_DEF_PRECISION_INT
+@deffnx Macro GAL_TABLE_DEF_PRECISION_FLT
+@deffnx Macro GAL_TABLE_DEF_PRECISION_DBL
+@cindex @code{printf}
+The default width and precision for generic types to use in writing numeric
+types into a text file (plain text and FITS ASCII tables). When the dataset
+doesn't have any pre-set width and precision (see @code{disp_width} and
+@code{disp_precision} in @ref{Generic data container}) these will be
+directly used in C's @code{printf} command to write the number as a string.
+@end deffn
-@deftypefun uint8_t gal_fits_bitpix_to_type (int @code{bitpix})
-Return the Gnuastro type identifier that corresponds to CFITSIO's
-@code{bitpix} on this system.
-@end deftypefun
+@deffn Macro GAL_TABLE_DISPLAY_FMT_STRING
+@deffnx Macro GAL_TABLE_DISPLAY_FMT_DECIMAL
+@deffnx Macro GAL_TABLE_DISPLAY_FMT_UDECIMAL
+@deffnx Macro GAL_TABLE_DISPLAY_FMT_OCTAL
+@deffnx Macro GAL_TABLE_DISPLAY_FMT_HEX
+@deffnx Macro GAL_TABLE_DISPLAY_FMT_FLOAT
+@deffnx Macro GAL_TABLE_DISPLAY_FMT_EXP
+@deffnx Macro GAL_TABLE_DISPLAY_FMT_GENERAL
+The display format used in C's @code{printf} to display data of different
+types. The @code{_STRING} and @code{_DECIMAL} are unique for printing
+strings and signed integers, they are mainly here for
+completeness. However, unsigned integers and floating points can be
+displayed in multiple formats:
-@deftypefun int gal_fits_type_to_bitpix (uint8_t @code{type})
-Return the CFITSIO @code{bitpix} value that corresponds to Gnuastro's
-@code{type}.
-@end deftypefun
+@table @asis
+@item Unsigned integer
+For unsigned integers, it is possible to choose from @code{_UDECIMAL}
+(unsigned decimal), @code{_OCTAL} (octal notation, for example @code{125}
+in decimal will be displayed as @code{175}), and @code{_HEX} (hexadecimal
+notation, for example @code{125} in decimal will be displayed as
+@code{7D}).
-@deftypefun char gal_fits_type_to_bin_tform (uint8_t @code{type})
-Return the FITS standard binary table @code{TFORM} character that
-corresponds to Gnuastro's @code{type}.
-@end deftypefun
+@item Floating point
+For floating point, it is possible to display the number in @code{_FLOAT}
+(floating point, for example @code{1500.345}), @code{_EXP} (exponential,
+for example @code{1.500345e+03}), or @code{_GENERAL} which is the best of
+the two for the given number.
+@end table
+@end deffn
-@deftypefun int gal_fits_type_to_datatype (uint8_t @code{type})
-Return the CFITSIO @code{datatype} that corresponds to Gnuastro's
-@code{type} on this machine.
-@end deftypefun
+@deffn Macro GAL_TABLE_FORMAT_INVALID
+@deffnx Macro GAL_TABLE_FORMAT_TXT
+@deffnx Macro GAL_TABLE_FORMAT_AFITS
+@deffnx Macro GAL_TABLE_FORMAT_BFITS
+All the current acceptable table formats to Gnuastro. The @code{AFITS} and
+@code{BFITS} represent FITS ASCII tables and FITS Binary tables. You can
+use these anywhere you see the @code{tableformat} variable.
+@end deffn
-@deftypefun uint8_t gal_fits_datatype_to_type (int @code{datatype}, int
@code{is_table_column})
-Return Gnuastro's type identifier that corresponds to the CFITSIO
-@code{datatype}. Note that when dealing with CFITSIO's @code{TLONG}, the
-fixed width type differs between tables and images. So if the corresponding
-dataset is a table column, put a non-zero value into
-@code{is_table_column}.
-@end deftypefun
+@deffn Macro GAL_TABLE_SEARCH_INVALID
+@deffnx Macro GAL_TABLE_SEARCH_NAME
+@deffnx Macro GAL_TABLE_SEARCH_UNIT
+@deffnx Macro GAL_TABLE_SEARCH_COMMENT
+When the desired column is not a number, these values determine if the
+string to match, or regular expression to search, be in the @emph{name},
+@emph{units} or @emph{comments} of the column meta data. These values
+should be used for the @code{searchin} variables of the functions.
+@end deffn
-@node FITS HDUs, FITS header keywords, CFITSIO and Gnuastro types, FITS files
-@subsubsection FITS HDUs
+@deftypefun {gal_data_t *} gal_table_info (char @code{*filename}, char
@code{*hdu}, size_t @code{*numcols}, size_t @code{*numrows}, int
@code{*tableformat})
+Store the information of each column in a table (either as a text file or
+as a FITS table) into an array of data structures with @code{numcols}
+structures (one data structure for each column). The number of rows is
+stored in the space that @code{numrows} points to. The format of the table
+(e.g., ascii text file, or FITS binary or ASCII table) will be put in
+@code{tableformat} (macros defined above). If the filename is not a FITS
+file, then @code{hdu} will not be used (can be @code{NULL}).
-A FITS file can contain multiple HDUs/extensions. The functions in this
-section can be used to get basic information about the extensions or open
-them. Note that @code{fitsfile} is defined in CFITSIO's @code{fitsio.h}
-which is automatically included by Gnuastro's @file{gnuastro/fits.h}.
+Note that other than the character strings (column name, units and
+comments), nothing in the data structure(s) will be allocated by this
+function for the actual data (e.g., the `array' or `dsize' elements). This
+function is just for column information (meta-data), not column contents.
+@end deftypefun
+
+@deftypefun void gal_table_print_info (gal_data_t @code{*allcols}, size_t
@code{numcols}, size_t @code{numrows})
+This program will print the column information for all the columns (output
+of @code{gal_table_info}). The output is in the same format as this command
+with Gnuastro Table program (see @ref{Table}):
+@example
+$ asttable --info table.fits
+@end example
+@end deftypefun
+
+@deftypefun {gal_data_t *} gal_table_read (char @code{*filename}, char
@code{*hdu}, gal_list_str_t @code{*cols}, int @code{searchin}, int
@code{ignorecase}, int @code{minmapsize}, size_t @code{colmatch})
+Read the specified columns in a text file (named @code{filename}) into a
+linked list of data structures. If the file is FITS, then @code{hdu} will
+also be used, otherwise, @code{hdu} is ignored.
+
+@cindex AWK
+@cindex GNU AWK
+The information to search for columns should be specified by the
+@code{cols} list of strings (see @ref{List of strings}). The string in each
+node of the list may be a number, an exact match to a column name, or a
+regular expression (in GNU AWK format) enclosed in @code{/ /}. The
+@code{searchin} value must be one of the macros defined above. If
+@code{cols} is NULL, then this function will read the full table.
+
+The output is an individually allocated list of datasets (see @ref{List of
+gal_data_t}) with the same order of the @code{cols} list. Note that one
+column node in the @code{cols} list might give multiple columns (for
+example from regular expressions), in this case, the order of output
+columns that correspond to that one input, are in order of the table (which
+column was read first). So the first requested column is the first popped
+data structure and so on.
+
+if @code{colmatch!=NULL}, it is assumed to be an array that has at least the
+same number of elements as nodes in the @code{cols} list. The number of
+columns that matched each input column will be stored in each element.
+@end deftypefun
+
+@cindex Git
+@deftypefun void gal_table_comments_add_intro (gal_list_str_t
@code{**comments}, char @code{*program_string}, time_t @code{*rawtime})
+Add some basic information to the list of @code{comments}. This basic
+information includes the following information
+@itemize
+@item
+If the program is run in a Git version controlled directory, Git's
+description is printed (see description under @code{COMMIT} in @ref{Output
+headers}).
+@item
+The calendar time that is stored in @code{rawtime} (@code{time_t} is C's
+calendar time format defined in @file{time.h}). You can calculate the time
+in this format with the following expressions:
+@example
+time_t rawtime;
+time(&rawtime);
+@end example
+@item
+The name of your program in @code{program_string}. If it is @code{NULL},
+this line is ignored.
+@end itemize
+@end deftypefun
+
+@deftypefun void gal_table_write (gal_data_t @code{*cols}, gal_list_str_t
@code{*comments}, int @code{tableformat}, char @code{*filename}, char
@code{*extname})
+Write the @code{cols} list of datasets into a table in @code{filename} (see
+@ref{List of gal_data_t}). The format of the table can be determined with
+@code{tableformat} that accepts the macros defined above. If
+@code{comments} is not @code{NULL}, then the list of comments will also be
+printed into the output table. When the output table is a plain text file,
+each node's string will be printed after a @code{#} (so it can be
+considered as a comment) and in FITS table they will follow a
+@code{COMMENT} keyword. If @file{filename} is a FITS file, the table
+extension that will be written will have the name @code{extname}.
+
+If a file named @file{filename} already exists, the operation depends on
+the type of output. When @file{filename} is a FITS file, the table will be
+added as a new extension after all existing ones. If @file{filename} is a
+plain text file, this function will abort with an error.
+@end deftypefun
+
+@deftypefun void gal_table_write_log (gal_data_t @code{*logll}, char
@code{*program_string}, time_t @code{*rawtime}, gal_list_str_t
@code{*comments}, char @code{*filename}, int @code{quiet})
+Write the @code{logll} list of datasets into a table in @code{filename}
+(see @ref{List of gal_data_t}). This function is just a wrapper around
+@code{gal_table_comments_add_intro} and @code{gal_table_write} (see
+above). If @code{quiet} is non-zero, this function will print a message
+saying that the @code{filename} has been created.
+@end deftypefun
+
+
+
+
+
+@node FITS files, File input output, Table input output, Gnuastro library
+@subsection FITS files (@file{fits.h})
+
+@cindex FITS
+@cindex CFITSIO
+The FITS format is the most common format to store data (images and tables)
+in astronomy. The CFITSIO library already provides a very good low-level
+collection of functions for manipulating FITS data. The low-level nature of
+CFITSIO is defined for versatility and portability. As a result, even a
+simple and basic operation, like reading an image or table column into
+memory, will require a special sequence of CFITSIO function calls which can
+be inconvenient and buggy to manage in separate locations. To ease this
+process, Gnuastro's library provides wrappers for CFITSIO functions. With
+these, it much easier to read, write, or modify FITS file data, header
+keywords and extensions. Hence, if you feel these functions don't exactly
+do what you want, we strongly recommend reading the CFITSIO manual to use
+its great features directly (afterwards, send us your wrappers so we can
+include it here for others to benefit also).
+
+All the functions and macros introduced in this section are declared in
+@file{gnuastro/fits.h}. When you include this header, you are also
+including CFITSIO's @file{fitsio.h} header. So you don't need to explicitly
+include @file{fitsio.h} anymore and can freely use any of its macros or
+functions in your code along with those discussed here.
+
+
+@menu
+* FITS macros errors filenames:: General macros, errors and checking names.
+* CFITSIO and Gnuastro types:: Conversion between FITS and Gnuastro types.
+* FITS HDUs:: Opening and getting information about HDUs.
+* FITS header keywords:: Reading and writing FITS header keywords.
+* FITS arrays:: Reading and writing FITS images/arrays.
+* FITS tables:: Reading and writing FITS tables.
+@end menu
+
+@node FITS macros errors filenames, CFITSIO and Gnuastro types, FITS files,
FITS files
+@subsubsection FITS Macros, errors and filenames
+
+Some general constructs provided by Gnuastro's FITS handling functions are
+discussed here. In particular there are several useful functions about FITS
+file names.
+
+@deffn Macro GAL_FITS_MAX_NDIM
+The maximum number of dimensions a dataset can have in FITS format,
+according to the FITS standard this is 999.
+@end deffn
+
+@deftypefun void gal_fits_io_error (int @code{status}, char @code{*message})
+If @code{status} is non-zero, this function will print the CFITSIO error
+message corresponding to status, print @code{message} (optional) in the
+next line and abort the program. If @code{message==NULL}, it will print a
+default string after the CFITSIO error.
+@end deftypefun
+
+@deftypefun int gal_fits_name_is_fits (char @code{*name})
+If the @code{name} is an acceptable CFITSIO FITS filename return @code{1}
+(one), otherwise return @code{0} (zero). The currently acceptable FITS
+suffixes are @file{.fits}, @file{.fit}, @file{.fits.gz}, @file{.fits.Z},
+@file{.imh}, @file{.fits.fz}. IMH is the IRAF format which is acceptable to
+CFITSIO.
+@end deftypefun
+
+@deftypefun int gal_fits_suffix_is_fits (char @code{*suffix})
+Similar to @code{gal_fits_name_is_fits}, but only for the suffix. The
+suffix doesn't have to start with `@key{.}': this function will return
+@code{1} (one) for both @code{fits} and @code{.fits}.
+@end deftypefun
+
+@deftypefun {char *} gal_fits_name_save_as_string (char @code{*filename}, char
@code{*hdu})
+If the name is a FITS name, then put a @code{(hdu: ...)} after it and
+return the string. If it isn't a FITS file, just print the name. Note that
+the space is allocated. This function is useful when you want to report a
+random file to the user which may be FITS or not (for a FITS file, simply
+the filename is not enough, the HDU is also necessary).
+@end deftypefun
+
+
+@node CFITSIO and Gnuastro types, FITS HDUs, FITS macros errors filenames,
FITS files
+@subsubsection CFITSIO and Gnuastro types
+
+Both Gnuastro and CFITSIO have special identifiers for each type that they
+accept. Gnuastro's type identifiers are fully described in @ref{Library
+data types} and are usable for all kinds of datasets (images, table columns
+and etc) as part of Gnuastro's @ref{Generic data container}. However,
+following the FITS standard, CFITSIO has different identifiers for images
+and tables. Following CFITSIO's own convention, we will use @code{bitpix}
+for image type identifiers and @code{datatype} for its internal identifiers
+(and mainly used in tables). The functions introduced in this section can
+be used to convert between CFITSIO and Gnuastro's type identifiers.
+
+One important issue to consider is that CFITSIO's types are not fixed width
+(for example @code{long} may be 32-bits or 64-bits on different
+systems). However, Gnuastro's types are defined by their width. These
+functions will use information on the host system to do the proper
+conversion, so it strongly recommended to use these functions for
+portability of your code and not to assume a fixed correspondence between
+CFITSIO and Gnuastro's types.
+
+@deftypefun uint8_t gal_fits_bitpix_to_type (int @code{bitpix})
+Return the Gnuastro type identifier that corresponds to CFITSIO's
+@code{bitpix} on this system.
+@end deftypefun
+
+@deftypefun int gal_fits_type_to_bitpix (uint8_t @code{type})
+Return the CFITSIO @code{bitpix} value that corresponds to Gnuastro's
+@code{type}.
+@end deftypefun
+
+@deftypefun char gal_fits_type_to_bin_tform (uint8_t @code{type})
+Return the FITS standard binary table @code{TFORM} character that
+corresponds to Gnuastro's @code{type}.
+@end deftypefun
+
+@deftypefun int gal_fits_type_to_datatype (uint8_t @code{type})
+Return the CFITSIO @code{datatype} that corresponds to Gnuastro's
+@code{type} on this machine.
+@end deftypefun
+
+@deftypefun uint8_t gal_fits_datatype_to_type (int @code{datatype}, int
@code{is_table_column})
+Return Gnuastro's type identifier that corresponds to the CFITSIO
+@code{datatype}. Note that when dealing with CFITSIO's @code{TLONG}, the
+fixed width type differs between tables and images. So if the corresponding
+dataset is a table column, put a non-zero value into
+@code{is_table_column}.
+@end deftypefun
+
+@node FITS HDUs, FITS header keywords, CFITSIO and Gnuastro types, FITS files
+@subsubsection FITS HDUs
+
+A FITS file can contain multiple HDUs/extensions. The functions in this
+section can be used to get basic information about the extensions or open
+them. Note that @code{fitsfile} is defined in CFITSIO's @code{fitsio.h}
+which is automatically included by Gnuastro's @file{gnuastro/fits.h}.
@deftypefun {fitsfile *} gal_fits_open_to_write (char @code{*filename})
If @file{filename} exists, open it and return the @code{fitsfile} pointer
@@ -21950,7 +23238,7 @@ then if the image has a name and units, the respective
string will be put
in these pointers.
@end deftypefun
-@deftypefun {gal_data_t *} gal_fits_img_read (char @code{*filename}, char
@code{*hdu}, size_t @code{minmapsize}, size_t @code{hstartwcs}, size_t
@code{hendwcs})
+@deftypefun {gal_data_t *} gal_fits_img_read (char @code{*filename}, char
@code{*hdu}, size_t @code{minmapsize})
Read the contents of the @code{hdu} extension/HDU of @code{filename} into a
Gnuastro generic data container (see @ref{Generic data container}) and
return it. If the necessary space is larger than @code{minmapsize}, then
@@ -21958,14 +23246,18 @@ don't keep the data in RAM, but in a file on the
HDD/SSD. For more on
@code{minmapsize} see the description under the same name in @ref{Generic
data container}.
-The @code{hstartwcs} and @code{hendwcs} arguments are the starting and
-ending header keywords to search for WCS information, if no limit is
-needed, set them to zero. For more on these two options, please see the
-description of @code{gal_wcs_read_fitsptr} in @ref{World Coordinate
-System}.
+Note that this function only reads the main data within the requested FITS
+extension, the WCS will not be read into the returned dataset. To read the
+WCS, you can use @code{gal_wcs_read} function as shown below. Afterwards,
+the @code{gal_data_free} function will free both the dataset and any WCS
+structure (if there are any).
+@example
+data=gal_fits_img_read(filename, hdu, -1);
+data->wcs=gal_wcs_read(filename, hdu, 0, 0, &data->wcs->nwcs);
+@end example
@end deftypefun
-@deftypefun {gal_data_t *} gal_fits_img_read_to_type (char @code{*inputname},
char @code{*inhdu}, uint8_t @code{type}, size_t @code{minmapsize}, )
+@deftypefun {gal_data_t *} gal_fits_img_read_to_type (char @code{*inputname},
char @code{*inhdu}, uint8_t @code{type}, size_t @code{minmapsize})
Read the contents of the @code{hdu} extension/HDU of @code{filename} into a
Gnuastro generic data container (see @ref{Generic data container}) of type
@code{type} and return it.
@@ -22111,165 +23403,32 @@ formats, see @ref{Table input output}.
+@node File input output, World Coordinate System, FITS files, Gnuastro library
+@subsection File input output
+The most commonly used file format in astronomical data analysis is the
+FITS format (see @ref{Fits} for an introduction), therefore Gnuastro's
+library provides a large and separate collection of functions to read/write
+data from/to them (see @ref{FITS files}). However, FITS is not well
+recognized outside the astronomical community and cannot be imported into
+documents or slides. Therefore, in this section, we discuss the other
+different file formats that Gnuastro's library recognizes.
+@menu
+* Text files:: Reading and writing from/to plain text files.
+* TIFF files:: Reading and writing from/to TIFF files.
+* JPEG files:: Reading and writing from/to JPEG files.
+* EPS files:: Writing to EPS files.
+* PDF files:: Writing to PDF files.
+@end menu
+@node Text files, TIFF files, File input output, File input output
+@subsubsection Text files (@file{txt.h})
-
-@node World Coordinate System, Text files, FITS files, Gnuastro library
-@subsection World Coordinate System (@file{wcs.h})
-
-The FITS standard defines the world coordinate system (WCS) as a mechanism
-to associate physical values to positions within a dataset. For example, it
-can be used to convert pixel coordinates in an image to celestial
-coordinates like the right ascension and declination. The functions in this
-section are mainly just wrappers over CFITSIO, WCSLIB and GSL library
-functions to help in common applications.
-
-
-@deftypefun {struct wcsprm *} gal_wcs_read_fitsptr (fitsfile @code{*fptr},
size_t @code{hstartwcs}, size_t @code{hendwcs}, int @code{*nwcs})
-[@strong{Not thread-safe}] Return the WCSLIB @code{wcsprm} structure that
-is read from the CFITSIO @code{fptr} pointer to an opened FITS file. Also
-put the number of coordinate representations found into the space that
-@code{nwcs} points to. To read the WCS structure directly from a filename,
-see @code{gal_wcs_read} below. After processing has finished, you can free
-the returned structure with WCSLIB's @code{wcsvfree} keyword:
-
-@example
-status = wcsvfree(&nwcs,&wcs);
-@end example
-
-If you don't want to search the full FITS header for WCS-related FITS
-keywords (for example due to conflicting keywords), but only a specific
-range of the header keywords you can use the @code{hstartwcs} and
-@code{hendwcs} arguments to specify the keyword number range (counting from
-zero). If @code{hendwcs} is larger than @code{hstartwcs}, then only
-keywords in the given range will be checked. Hence, to ignore this feature
-(and search the full FITS header), give both these arguments the same
-value.
-
-If the WCS information couldn't be read from the FITS file, this function
-will return a @code{NULL} pointer and put a zero in @code{nwcs}. A WCSLIB
-error message will also be printed in @code{stderr} if there was an error.
-
-This function is just a wrapper over WCSLIB's @code{wcspih} function which
-is not thread-safe. Therefore, be sure to not call this function
-simultaneously (over multiple threads).
-@end deftypefun
-
-@deftypefun {struct wcsprm *} gal_wcs_read (char @code{*filename}, char
@code{*hdu}, size_t @code{hstartwcs}, size_t @code{hendwcs}, int @code{*nwcs})
-[@strong{Not thread-safe}] Return the WCSLIB structure that is read from
-the HDU/extension @code{hdu} of the file @code{filename}. Also put the
-number of coordinate representations found into the space that @code{nwcs}
-points to. Please see @code{gal_wcs_read_fitsptr} for more.
-@end deftypefun
-
-@deftypefun {struct wcsprm *} gal_wcs_copy (struct wcsprm @code{*wcs})
-Return a fully allocated (independent) copy of @code{wcs}.
-@end deftypefun
-
-@deftypefun void gal_wcs_on_tile (gal_data_t @code{*tile})
-Create a WCSLIB @code{wcsprm} structure for @code{tile} using WCS
-parameters of the tile's allocated block dataset, see @ref{Tessellation
-library} for the definition of tiles. If @code{tile} already has a WCS
-structure, this function won't do anything.
-
-In many cases, tiles are created for internal/low-level processing. Hence
-for performance reasons, when creating the tiles they don't have any WCS
-structure. When needed, this function can be used to add a WCS structure to
-each tile tile by copying the WCS structure of its block and correcting the
-reference point's coordinates within the tile.
-@end deftypefun
-
-@deftypefun {double *} gal_wcs_warp_matrix (struct wcsprm @code{*wcs})
-Return the Warping matrix of the given WCS structure as an array of double
-precision floating points. This will be the final matrix, irrespective of
-the type of storage in the WCS structure. Recall that the FITS standard has
-several methods to store the matrix. The output is an allocated square
-matrix with each side equal to the number of dimensions.
-@end deftypefun
-
-@deftypefun void gal_wcs_decompose_pc_cdelt (struct wcsprm @code{*wcs})
-Decompose the @code{PCi_j} and @code{CDELTi} elements of
-@code{wcs}. According to the FITS standard, in the @code{PCi_j} WCS
-formalism, the rotation matrix elements @mymath{m_{ij}} are encoded in the
-@code{PCi_j} keywords and the scale factors are encoded in the
-@code{CDELTi} keywords. There is also another formalism (the @code{CDi_j}
-formalism) which merges the two into one matrix.
-
-However, WCSLIB's internal operations are apparently done in the
-@code{PCi_j} formalism. So its outputs are also all in that format by
-default. When the input is a @code{CDi_j}, WCSLIB will still read the
-matrix directly into the @code{PCi_j} matrix and the @code{CDELTi} values
-are set to @code{1} (one). This function is designed to correct such
-issues: after it is finished, the @code{CDELTi} values in @code{wcs} will
-correspond to the pixel scale, and the @code{PCi_j} will correction show
-the rotation.
-@end deftypefun
-
-@deftypefun double gal_wcs_angular_distance_deg (double @code{r1}, double
@code{d1}, double @code{r2}, double @code{d2})
-Return the angular distance (in degrees) between a point located at
-(@code{r1}, @code{d1}) to (@code{r2}, @code{d2}). All input coordinates are
-in degrees. The distance (along a great circle) on a sphere between two
-points is calculated with the equation below.
-
-@dispmath {\cos(d)=\sin(d_1)\sin(d_2)+\cos(d_1)\cos(d_2)\cos(r_1-r_2)}
-
-However, since the the pixel scales are usually very small numbers, this
-function won't use that direct formula. It will be use the
-@url{https://en.wikipedia.org/wiki/Haversine_formula, Haversine formula}
-which is better considering floating point errors:
-
-@dispmath{{\sin^2(d)\over 2}=\sin^2\left( {d_1-d_2\over 2}
\right)+\cos(d_1)\cos(d_2)\sin^2\left( {r_1-r_2\over 2} \right)}
-@end deftypefun
-
-@deftypefun {double *} gal_wcs_pixel_scale (struct wcsprm @code{*wcs})
-Return the pixel scale for each dimension of @code{wcs} in degrees. The
-output is an array of double precision floating point type with one element
-for each dimension.
-@end deftypefun
-
-@deftypefun double gal_wcs_pixel_area_arcsec2 (struct wcsprm @code{*wcs})
-Return the pixel area of @code{wcs} in arcsecond squared. If the input WCS
-structure is not two dimensional and the units (@code{CUNIT} keywords) are
-not @code{deg} (for degrees), then this function will return a NaN.
-@end deftypefun
-
-@deftypefun {gal_data_t *} gal_wcs_world_to_img (gal_data_t @code{*coords},
struct wcsprm @code{*wcs}, int @code{inplace})
-Convert the linked list of world coordinates in @code{coords} to a linked
-list of image coordinates given the input WCS structure. @code{coords} must
-be a linked list of data structures of float64 (`double') type,
-see@ref{Linked lists} and @ref{List of gal_data_t}. The top (first
-popped/read) node of the linked list must be the first WCS coordinate (RA
-in an image usually) and etc. Similarly, the top node of the output will be
-the first image coordinate (in the FITS standard).
-
-If @code{inplace} is zero, then the output will be a newly allocated list
-and the input list will be untouched. However, if @code{inplace} is
-non-zero, the output values will be written into the input's already
-allocated array and the returned pointer will be the same pointer to
-@code{coords} (in other words, you can ignore the returned value). Note
-that in the latter case, only the values will be changed, things like units
-or name (if present) will be untouched.
-@end deftypefun
-
-@deftypefun {gal_data_t *} gal_wcs_img_to_world (gal_data_t @code{*coords},
struct wcsprm @code{*wcs}, int @code{inplace})
-Convert the linked list of image coordinates in @code{coords} to a linked
-list of world coordinates given the input WCS structure. See the
-description of @code{gal_wcs_world_to_img} for more details.
-@end deftypefun
-
-
-
-@node Text files, Table input output, World Coordinate System, Gnuastro library
-@subsection Text files (@file{txt.h})
-
-FITS files are the primary data container in astronomy. FITS indeed as many
-useful features, but the most universal and portable format for data
-storage are plain text files. They can be viewed and edited on any text
-editor or even on the command-line. Therefore the functions in this section
-are defined to simplify reading from and writing to plain text
-files.
+The most universal and portable format for data storage are plain text
+files. They can be viewed and edited on any text editor or even on the
+command-line. This section are describes some functions that help in
+reading from and writing to plain text files.
Lines are one of the most basic buiding blocks (delimiters) of a text
file. Some operating systems like Microsoft Windows, terminate their ASCII
@@ -22369,191 +23528,368 @@ formats, see @ref{Table input output}.
@end deftypefun
-@node Table input output, Arithmetic on datasets, Text files, Gnuastro library
-@subsection Table input output (@file{table.h})
+@node TIFF files, JPEG files, Text files, File input output
+@subsubsection TIFF files (@file{tiff.h})
+
+@cindex TIFF format
+Outside of astronomy, the TIFF standard is arguably the most commonly used
+format to store high-precision data/images. Unlike FITS however, the TIFF
+standard only supports images (not tables), but like FITS, it has support
+for all standard data types (see @ref{Numeric data types}) which is the
+primary reason other fields use it.
+
+Another similarity of the TIFF and FITS standards is that TIFF supports
+multiple images in one file. The TIFF standard calls each one of these
+images (and their accompanying meta-data) a `directory' (roughly equivalent
+to the FITS extensions). Unlike FITS however, the directories can only be
+identified by their number (counting from zero), recall that in FITS you
+can also use the extension name to identify it.
+
+The functions described here allow easy reading (and later writing) of TIFF
+files within Gnuastro or for users of Gnuastro's libraries. Currently only
+reading is supported, but if you are interested, please get in touch with
+us.
+
+@deftypefun {int} gal_tiff_name_is_tiff (char @code{*name})
+Return @code{1} if @code{name} has a TIFF suffix. This can be used to make
+sure that a given input file is TIFF. See @code{gal_tiff_suffix_is_tiff}
+for a list of recognized suffixes.
+@end deftypefun
-Tables are a collection of one dimensional datasets that are packed
-together into one file. They are the single most common format to store
-high-level (processed) information, hence they play a very important role
-in Gnuastro. For a more thorough introduction, please see
-@ref{Table}. Gnuastro's Table program, and all the other programs that can
-read from and write into tables, use the functions of this section for
-reading and writing their input/output tables. For a simple demonstration
-of using the constructs introduced here, see @ref{Library demo - reading
-and writing table columns}.
+@deftypefun {int} gal_tiff_suffix_is_tiff (char @code{*name})
+Return @code{1} if @code{suffix} is a recognized TIFF suffix. The
+recognized suffixes are @file{tif}, @file{tiff}, @file{TIFF} and
+@file{TIFF}.
+@end deftypefun
-Currently only plain text (see @ref{Gnuastro text table format}) and FITS
-(ASCII and binary) tables are supported by Gnuastro. However, the low-level
-table infra-structure is written such that accommodating other formats is
-also possible and in future releases more formats will hopefully be
-supported. Please don't hesitate to suggest your favorite format so it can
-be implemented when possible.
+@deftypefun {size_t} gal_tiff_dir_string_read (char @code{*string})
+Return the number within @code{string} as a @code{size_t} number to
+identify a TIFF directory. Note that the directories start counting from
+zero.
+@end deftypefun
-@deffn Macro GAL_TABLE_DEF_WIDTH_STR
-@deffnx Macro GAL_TABLE_DEF_WIDTH_INT
-@deffnx Macro GAL_TABLE_DEF_WIDTH_LINT
-@deffnx Macro GAL_TABLE_DEF_WIDTH_FLT
-@deffnx Macro GAL_TABLE_DEF_WIDTH_DBL
-@deffnx Macro GAL_TABLE_DEF_PRECISION_INT
-@deffnx Macro GAL_TABLE_DEF_PRECISION_FLT
-@deffnx Macro GAL_TABLE_DEF_PRECISION_DBL
-@cindex @code{printf}
-The default width and precision for generic types to use in writing numeric
-types into a text file (plain text and FITS ASCII tables). When the dataset
-doesn't have any pre-set width and precision (see @code{disp_width} and
-@code{disp_precision} in @ref{Generic data container}) these will be
-directly used in C's @code{printf} command to write the number as a string.
-@end deffn
+@deftypefun {gal_data_t *} gal_tiff_read (char @code{*filename}, size_t
@code{dir}, size_t @code{minmapsize})
+Read the @code{dir} directory within the TIFF file @code{filename} and
+return the contents of that TIFF directory as @code{gal_data_t}. If the
+directory's image contains multiple channels, the output will be a list
+(see @ref{List of gal_data_t}).
+@end deftypefun
+
+
+
+
+
+@node JPEG files, EPS files, TIFF files, File input output
+@subsubsection JPEG files (@file{jpeg.h})
+
+@cindex JPEG format
+The JPEG file format is one of the most common formats for storing and
+transferring images, recognized by almost all image rendering and
+processing programs. In particular, because of its lossy compression
+algorithm, JPEG files can have low volumes, making it used heavily on the
+internet. For more on this file format, and a comparison with others,
+please see @ref{Recognized file formats}.
+
+For scientific purposes, the lossy compression and very limited dymanic
+range (8-bit integers) make JPEG very un-attractive for storing of valuable
+data. However, because of its commonality, it will inevitably be needed in
+some situations. The functions here can be used to read and write JPEG
+images into Gnuastro's @ref{Generic data container}. If the JPEG file has
+more than one color channel, each channel is treated as a separate node in
+a list of datasets (see @ref{List of gal_data_t}).
+
+@deftypefun {int} gal_jpeg_name_is_jpeg (char @code{*name})
+Return @code{1} if @code{name} has a JPEG suffix. This can be used to make
+sure that a given input file is JPEG. See @code{gal_jpeg_suffix_is_jpeg}
+for a list of recognized suffixes.
+@end deftypefun
+
+@deftypefun {int} gal_jpeg_suffix_is_jpeg (char @code{*name})
+Return @code{1} if @code{suffix} is a recognized JPEG suffix. The
+recognized suffixes are @code{.jpg}, @code{.JPG}, @code{.jpeg},
+@code{.JPEG}, @code{.jpe}, @code{.jif}, @code{.jfif} and @code{.jfi}.
+@end deftypefun
+
+@deftypefun {gal_data_t *} gal_jpeg_read (char @code{*filename}, size_t
@code{minmapsize})
+Read the JPEG file @code{filename} and return the contents as
+@code{gal_data_t}. If the directory's image contains multiple
+colors/channels, the output will be a list with one node per color/channel
+(see @ref{List of gal_data_t}).
+@end deftypefun
+
+@cindex JPEG compression quality
+@deftypefun {void} gal_jpeg_write (gal_data_t @code{*in}, char
@code{*filename}, uint8_t @code{quality}, float @code{widthincm})
+Write the given dataset (@code{in}) into @file{filename} (a JPEG file). If
+@code{in} is a list, then each node in the list will be a color channel,
+therefore there can only be 1, 3 or 4 nodes in the list. If the number of
+nodes is different, then this function will abort the program with a
+message describing the cause. The lossy JPEG compression level can be set
+through @code{quality} which is a value between 0 and 100 (inclusive, 100
+being the best quality). The display width of the JPEG file in units of
+centimeters (to suggest to viewers/users, only a meta-data) can be set
+through @code{widthincm}.
+@end deftypefun
+
+
+
+
+
+@node EPS files, PDF files, JPEG files, File input output
+@subsubsection EPS files (@file{eps.h})
+
+The Encapsulated PostScript (EPS) format is commonly used to store images
+(or individual/single-page parts of a document) in the PostScript
+documents. For a more complete introduction, please see @ref{Recognized
+file formats}. To provide high quality graphics, the Postscript language is
+a vectorized format, therefore pixels (elements of a ``rasterized'' format)
+aren't defined in their context.
+
+To display rasterized images, PostScript does allow arrays of
+pixels. However, since the over-all EPS file may contain many vectorized
+elements (for example borders, text, or other lines over the text) and
+interpretting them is not trivial or necessary within Gnuastro's scope,
+Gnuastro only provides some functions to write a dataset (in the
+@code{gal_data_t} format, see @ref{Generic data container}) into EPS.
+
+@deftypefun {int} gal_eps_name_is_eps (char @code{*name})
+Return @code{1} if @code{name} has an EPS suffix. This can be used to make
+sure that a given input file is EPS. See @code{gal_eps_suffix_is_eps} for a
+list of recognized suffixes.
+@end deftypefun
+
+@deftypefun {int} gal_eps_suffix_is_eps (char @code{*name})
+Return @code{1} if @code{suffix} is a recognized EPS suffix. The recognized
+suffixes are @code{.eps}, @code{.EPS}, @code{.epsf}, @code{.epsi}.
+@end deftypefun
+
+@deftypefun {void} gal_eps_to_pt (float @code{widthincm}, size_t
@code{*dsize}, size_t @code{*w_h_in_pt})
+Given a specific width in centimeters (@code{widthincm} and the number of
+the dataset's pixels in each dimension (@code{dsize}) calculate the size of
+the output in PostScript points. The output values are written in the
+@code{w_h_in_pt} array (which has to be allocated before calling this
+function). The first element in @code{w_h_in_pt} is the width and the
+second is the height of the image.
+@end deftypefun
+
+@deftypefun {void} gal_eps_write (gal_data_t @code{*in}, char
@code{*filename}, float @code{widthincm}, uint32_t @code{borderwidth}, int
@code{hex}, int @code{forpdf})
+Write the @code{in} dataset into an EPS file called
+@code{filename}. @code{in} has to be an unsigned 8-bit character type
+(@code{GAL_TYPE_UINT8}, see @ref{Numeric data types}). The desired width of
+the image in human/non-pixel units (to help the displayer) can be set with
+the @code{widthincm} argument. If @code{borderwidth} is non-zero, it is
+interpretted as the width (in points) of a solid black border around the
+image. A border can helpful when importing the EPS file into a document.
+
+@cindex ASCII85 encoding
+@cindex Hexadecimal encoding
+EPS files are plain-text (can be opened/edited in a text editor), therefore
+there are different encodings to store the data (pixel values) within
+them. Gnuastro supports the Hexadecimal and ASCII85 encoding. ASCII85 is
+more efficient (producing small file sizes), so it is the default
+encoding. To use Hexademical encoding, set @code{hex} to a non-zero
+value. Currently If you don't directly want to import the EPS file into a
+PostScript document but want to later compile it into a PDF file, set the
+@code{forpdf} argument to @code{1}.
+@end deftypefun
+
+
+
+
+
+@node PDF files, , EPS files, File input output
+@subsubsection PDF files (@file{pdf.h})
+
+The portable document format (PDF) has arguably become the most common
+format used for distribution of documents. In practice, a PDF file is just
+a compiled PostScript file. For a more complete introduction, please see
+@ref{Recognized file formats}. To provide high quality graphics, the PDF is
+a vectorized format, therefore pixels (elements of a ``rasterized'' format)
+aren't defined in their context. As a result, similar to @ref{EPS files},
+Gnuastro only writes datasets to a PDF file, not vice-versa.
+
+@deftypefun {int} gal_pdf_name_is_pdf (char @code{*name})
+Return @code{1} if @code{name} has an PDF suffix. This can be used to make
+sure that a given input file is PDF. See @code{gal_pdf_suffix_is_pdf} for a
+list of recognized suffixes.
+@end deftypefun
+
+@deftypefun {int} gal_pdf_suffix_is_pdf (char @code{*name})
+Return @code{1} if @code{suffix} is a recognized PDF suffix. The recognized
+suffixes are @code{.pdf} and @code{.PDF}.
+@end deftypefun
+
+@deftypefun {void} gal_pdf_write (gal_data_t @code{*in}, char
@code{*filename}, float @code{widthincm}, uint32_t @code{borderwidth})
+Write the @code{in} dataset into an EPS file called
+@code{filename}. @code{in} has to be an unsigned 8-bit character type
+(@code{GAL_TYPE_UINT8}, see @ref{Numeric data types}). The desired width of
+the image in human/non-pixel units (to help the displayer) can be set with
+the @code{widthincm} argument. If @code{borderwidth} is non-zero, it is
+interpretted as the width (in points) of a solid black border around the
+image. A border can helpful when importing the PDF file into a
+document.
+
+This function is just a wrapper for the @code{gal_eps_write} function in
+@ref{EPS files}. After making the EPS file, Ghostscript (with a version of
+9.10 or above, see @ref{Optional dependencies}) will be used to compile the
+EPS file to a PDF file. Therfore if GhostScript doesn't exist, doesn't have
+the proper version, or fails for any other reason, the EPS file will
+remain. It can be used to find the cause, or use another converter or
+PostScript compiler.
+@end deftypefun
+
+
+
+
+
+
+@node World Coordinate System, Arithmetic on datasets, File input output,
Gnuastro library
+@subsection World Coordinate System (@file{wcs.h})
+
+The FITS standard defines the world coordinate system (WCS) as a mechanism
+to associate physical values to positions within a dataset. For example, it
+can be used to convert pixel coordinates in an image to celestial
+coordinates like the right ascension and declination. The functions in this
+section are mainly just wrappers over CFITSIO, WCSLIB and GSL library
+functions to help in common applications.
+
+
+@deftypefun {struct wcsprm *} gal_wcs_read_fitsptr (fitsfile @code{*fptr},
size_t @code{hstartwcs}, size_t @code{hendwcs}, int @code{*nwcs})
+[@strong{Not thread-safe}] Return the WCSLIB @code{wcsprm} structure that
+is read from the CFITSIO @code{fptr} pointer to an opened FITS file. Also
+put the number of coordinate representations found into the space that
+@code{nwcs} points to. To read the WCS structure directly from a filename,
+see @code{gal_wcs_read} below. After processing has finished, you can free
+the returned structure with WCSLIB's @code{wcsvfree} keyword:
+
+@example
+status = wcsvfree(&nwcs,&wcs);
+@end example
-@deffn Macro GAL_TABLE_DISPLAY_FMT_STRING
-@deffnx Macro GAL_TABLE_DISPLAY_FMT_DECIMAL
-@deffnx Macro GAL_TABLE_DISPLAY_FMT_UDECIMAL
-@deffnx Macro GAL_TABLE_DISPLAY_FMT_OCTAL
-@deffnx Macro GAL_TABLE_DISPLAY_FMT_HEX
-@deffnx Macro GAL_TABLE_DISPLAY_FMT_FLOAT
-@deffnx Macro GAL_TABLE_DISPLAY_FMT_EXP
-@deffnx Macro GAL_TABLE_DISPLAY_FMT_GENERAL
-The display format used in C's @code{printf} to display data of different
-types. The @code{_STRING} and @code{_DECIMAL} are unique for printing
-strings and signed integers, they are mainly here for
-completeness. However, unsigned integers and floating points can be
-displayed in multiple formats:
+If you don't want to search the full FITS header for WCS-related FITS
+keywords (for example due to conflicting keywords), but only a specific
+range of the header keywords you can use the @code{hstartwcs} and
+@code{hendwcs} arguments to specify the keyword number range (counting from
+zero). If @code{hendwcs} is larger than @code{hstartwcs}, then only
+keywords in the given range will be checked. Hence, to ignore this feature
+(and search the full FITS header), give both these arguments the same
+value.
-@table @asis
-@item Unsigned integer
-For unsigned integers, it is possible to choose from @code{_UDECIMAL}
-(unsigned decimal), @code{_OCTAL} (octal notation, for example @code{125}
-in decimal will be displayed as @code{175}), and @code{_HEX} (hexadecimal
-notation, for example @code{125} in decimal will be displayed as
-@code{7D}).
+If the WCS information couldn't be read from the FITS file, this function
+will return a @code{NULL} pointer and put a zero in @code{nwcs}. A WCSLIB
+error message will also be printed in @code{stderr} if there was an error.
-@item Floating point
-For floating point, it is possible to display the number in @code{_FLOAT}
-(floating point, for example @code{1500.345}), @code{_EXP} (exponential,
-for example @code{1.500345e+03}), or @code{_GENERAL} which is the best of
-the two for the given number.
-@end table
-@end deffn
+This function is just a wrapper over WCSLIB's @code{wcspih} function which
+is not thread-safe. Therefore, be sure to not call this function
+simultaneously (over multiple threads).
+@end deftypefun
-@deffn Macro GAL_TABLE_FORMAT_INVALID
-@deffnx Macro GAL_TABLE_FORMAT_TXT
-@deffnx Macro GAL_TABLE_FORMAT_AFITS
-@deffnx Macro GAL_TABLE_FORMAT_BFITS
-All the current acceptable table formats to Gnuastro. The @code{AFITS} and
-@code{BFITS} represent FITS ASCII tables and FITS Binary tables. You can
-use these anywhere you see the @code{tableformat} variable.
-@end deffn
+@deftypefun {struct wcsprm *} gal_wcs_read (char @code{*filename}, char
@code{*hdu}, size_t @code{hstartwcs}, size_t @code{hendwcs}, int @code{*nwcs})
+[@strong{Not thread-safe}] Return the WCSLIB structure that is read from
+the HDU/extension @code{hdu} of the file @code{filename}. Also put the
+number of coordinate representations found into the space that @code{nwcs}
+points to. Please see @code{gal_wcs_read_fitsptr} for more.
+@end deftypefun
-@deffn Macro GAL_TABLE_SEARCH_INVALID
-@deffnx Macro GAL_TABLE_SEARCH_NAME
-@deffnx Macro GAL_TABLE_SEARCH_UNIT
-@deffnx Macro GAL_TABLE_SEARCH_COMMENT
-When the desired column is not a number, these values determine if the
-string to match, or regular expression to search, be in the @emph{name},
-@emph{units} or @emph{comments} of the column meta data. These values
-should be used for the @code{searchin} variables of the functions.
-@end deffn
+@deftypefun {struct wcsprm *} gal_wcs_copy (struct wcsprm @code{*wcs})
+Return a fully allocated (independent) copy of @code{wcs}.
+@end deftypefun
-@deftypefun {gal_data_t *} gal_table_info (char @code{*filename}, char
@code{*hdu}, size_t @code{*numcols}, size_t @code{*numrows}, int
@code{*tableformat})
-Store the information of each column in a table (either as a text file or
-as a FITS table) into an array of data structures with @code{numcols}
-structures (one data structure for each column). The number of rows is
-stored in the space that @code{numrows} points to. The format of the table
-(e.g., ascii text file, or FITS binary or ASCII table) will be put in
-@code{tableformat} (macros defined above). If the filename is not a FITS
-file, then @code{hdu} will not be used (can be @code{NULL}).
+@deftypefun void gal_wcs_on_tile (gal_data_t @code{*tile})
+Create a WCSLIB @code{wcsprm} structure for @code{tile} using WCS
+parameters of the tile's allocated block dataset, see @ref{Tessellation
+library} for the definition of tiles. If @code{tile} already has a WCS
+structure, this function won't do anything.
-Note that other than the character strings (column name, units and
-comments), nothing in the data structure(s) will be allocated by this
-function for the actual data (e.g., the `array' or `dsize' elements). This
-function is just for column information (meta-data), not column contents.
+In many cases, tiles are created for internal/low-level processing. Hence
+for performance reasons, when creating the tiles they don't have any WCS
+structure. When needed, this function can be used to add a WCS structure to
+each tile tile by copying the WCS structure of its block and correcting the
+reference point's coordinates within the tile.
@end deftypefun
-@deftypefun void gal_table_print_info (gal_data_t @code{*allcols}, size_t
@code{numcols}, size_t @code{numrows})
-This program will print the column information for all the columns (output
-of @code{gal_table_info}). The output is in the same format as this command
-with Gnuastro Table program (see @ref{Table}):
-@example
-$ asttable --info table.fits
-@end example
+@deftypefun {double *} gal_wcs_warp_matrix (struct wcsprm @code{*wcs})
+Return the Warping matrix of the given WCS structure as an array of double
+precision floating points. This will be the final matrix, irrespective of
+the type of storage in the WCS structure. Recall that the FITS standard has
+several methods to store the matrix. The output is an allocated square
+matrix with each side equal to the number of dimensions.
@end deftypefun
-@deftypefun {gal_data_t *} gal_table_read (char @code{*filename}, char
@code{*hdu}, gal_list_str_t @code{*cols}, int @code{searchin}, int
@code{ignorecase}, int @code{minmapsize}, size_t @code{colmatch})
-Read the specified columns in a text file (named @code{filename}) into a
-linked list of data structures. If the file is FITS, then @code{hdu} will
-also be used, otherwise, @code{hdu} is ignored.
+@deftypefun void gal_wcs_decompose_pc_cdelt (struct wcsprm @code{*wcs})
+Decompose the @code{PCi_j} and @code{CDELTi} elements of
+@code{wcs}. According to the FITS standard, in the @code{PCi_j} WCS
+formalism, the rotation matrix elements @mymath{m_{ij}} are encoded in the
+@code{PCi_j} keywords and the scale factors are encoded in the
+@code{CDELTi} keywords. There is also another formalism (the @code{CDi_j}
+formalism) which merges the two into one matrix.
-@cindex AWK
-@cindex GNU AWK
-The information to search for columns should be specified by the
-@code{cols} list of strings (see @ref{List of strings}). The string in each
-node of the list may be a number, an exact match to a column name, or a
-regular expression (in GNU AWK format) enclosed in @code{/ /}. The
-@code{searchin} value must be one of the macros defined above. If
-@code{cols} is NULL, then this function will read the full table.
+However, WCSLIB's internal operations are apparently done in the
+@code{PCi_j} formalism. So its outputs are also all in that format by
+default. When the input is a @code{CDi_j}, WCSLIB will still read the
+matrix directly into the @code{PCi_j} matrix and the @code{CDELTi} values
+are set to @code{1} (one). This function is designed to correct such
+issues: after it is finished, the @code{CDELTi} values in @code{wcs} will
+correspond to the pixel scale, and the @code{PCi_j} will correction show
+the rotation.
+@end deftypefun
-The output is an individually allocated list of datasets (see @ref{List of
-gal_data_t}) with the same order of the @code{cols} list. Note that one
-column node in the @code{cols} list might give multiple columns (for
-example from regular expressions), in this case, the order of output
-columns that correspond to that one input, are in order of the table (which
-column was read first). So the first requested column is the first popped
-data structure and so on.
+@deftypefun double gal_wcs_angular_distance_deg (double @code{r1}, double
@code{d1}, double @code{r2}, double @code{d2})
+Return the angular distance (in degrees) between a point located at
+(@code{r1}, @code{d1}) to (@code{r2}, @code{d2}). All input coordinates are
+in degrees. The distance (along a great circle) on a sphere between two
+points is calculated with the equation below.
-if @code{colmatch!=NULL}, it is assumed to be an array that has at least the
-same number of elements as nodes in the @code{cols} list. The number of
-columns that matched each input column will be stored in each element.
+@dispmath {\cos(d)=\sin(d_1)\sin(d_2)+\cos(d_1)\cos(d_2)\cos(r_1-r_2)}
+
+However, since the the pixel scales are usually very small numbers, this
+function won't use that direct formula. It will be use the
+@url{https://en.wikipedia.org/wiki/Haversine_formula, Haversine formula}
+which is better considering floating point errors:
+
+@dispmath{{\sin^2(d)\over 2}=\sin^2\left( {d_1-d_2\over 2}
\right)+\cos(d_1)\cos(d_2)\sin^2\left( {r_1-r_2\over 2} \right)}
@end deftypefun
-@cindex Git
-@deftypefun void gal_table_comments_add_intro (gal_list_str_t
@code{**comments}, char @code{*program_string}, time_t @code{*rawtime})
-Add some basic information to the list of @code{comments}. This basic
-information includes the following information
-@itemize
-@item
-If the program is run in a Git version controlled directory, Git's
-description is printed (see description under @code{COMMIT} in @ref{Output
-headers}).
-@item
-The calendar time that is stored in @code{rawtime} (@code{time_t} is C's
-calendar time format defined in @file{time.h}). You can calculate the time
-in this format with the following expressions:
-@example
-time_t rawtime;
-time(&rawtime);
-@end example
-@item
-The name of your program in @code{program_string}. If it is @code{NULL},
-this line is ignored.
-@end itemize
+@deftypefun {double *} gal_wcs_pixel_scale (struct wcsprm @code{*wcs})
+Return the pixel scale for each dimension of @code{wcs} in degrees. The
+output is an array of double precision floating point type with one element
+for each dimension.
@end deftypefun
-@deftypefun void gal_table_write (gal_data_t @code{*cols}, gal_list_str_t
@code{*comments}, int @code{tableformat}, char @code{*filename}, char
@code{*extname})
-Write the @code{cols} list of datasets into a table in @code{filename} (see
-@ref{List of gal_data_t}). The format of the table can be determined with
-@code{tableformat} that accepts the macros defined above. If
-@code{comments} is not @code{NULL}, then the list of comments will also be
-printed into the output table. When the output table is a plain text file,
-each node's string will be printed after a @code{#} (so it can be
-considered as a comment) and in FITS table they will follow a
-@code{COMMENT} keyword. If @file{filename} is a FITS file, the table
-extension that will be written will have the name @code{extname}.
+@deftypefun double gal_wcs_pixel_area_arcsec2 (struct wcsprm @code{*wcs})
+Return the pixel area of @code{wcs} in arcsecond squared. If the input WCS
+structure is not two dimensional and the units (@code{CUNIT} keywords) are
+not @code{deg} (for degrees), then this function will return a NaN.
+@end deftypefun
-If a file named @file{filename} already exists, the operation depends on
-the type of output. When @file{filename} is a FITS file, the table will be
-added as a new extension after all existing ones. If @file{filename} is a
-plain text file, this function will abort with an error.
+@deftypefun {gal_data_t *} gal_wcs_world_to_img (gal_data_t @code{*coords},
struct wcsprm @code{*wcs}, int @code{inplace})
+Convert the linked list of world coordinates in @code{coords} to a linked
+list of image coordinates given the input WCS structure. @code{coords} must
+be a linked list of data structures of float64 (`double') type,
+see@ref{Linked lists} and @ref{List of gal_data_t}. The top (first
+popped/read) node of the linked list must be the first WCS coordinate (RA
+in an image usually) and etc. Similarly, the top node of the output will be
+the first image coordinate (in the FITS standard).
+
+If @code{inplace} is zero, then the output will be a newly allocated list
+and the input list will be untouched. However, if @code{inplace} is
+non-zero, the output values will be written into the input's already
+allocated array and the returned pointer will be the same pointer to
+@code{coords} (in other words, you can ignore the returned value). Note
+that in the latter case, only the values will be changed, things like units
+or name (if present) will be untouched.
@end deftypefun
-@deftypefun void gal_table_write_log (gal_data_t @code{*logll}, char
@code{*program_string}, time_t @code{*rawtime}, gal_list_str_t
@code{*comments}, char @code{*filename}, int @code{quiet})
-Write the @code{logll} list of datasets into a table in @code{filename}
-(see @ref{List of gal_data_t}). This function is just a wrapper around
-@code{gal_table_comments_add_intro} and @code{gal_table_write} (see
-above). If @code{quiet} is non-zero, this function will print a message
-saying that the @code{filename} has been created.
+@deftypefun {gal_data_t *} gal_wcs_img_to_world (gal_data_t @code{*coords},
struct wcsprm @code{*wcs}, int @code{inplace})
+Convert the linked list of image coordinates in @code{coords} to a linked
+list of world coordinates given the input WCS structure. See the
+description of @code{gal_wcs_world_to_img} for more details.
@end deftypefun
-@node Arithmetic on datasets, Tessellation library, Table input output,
Gnuastro library
+
+
+
+
+@node Arithmetic on datasets, Tessellation library, World Coordinate System,
Gnuastro library
@subsection Arithmetic on datasets (@file{arithmetic.h})
When the dataset's type and other information are already known, any
@@ -23312,7 +24648,7 @@ char *filename="input.fits", *hdu="1";
...
/* Read the input dataset. */
-input=gal_fits_img_read(filename, hdu, -1, 0, 0);
+input=gal_fits_img_read(filename, hdu, -1);
/* Do a sanity check and preparations. */
gal_tile_full_sanity_check(filename, hdu, input, &tl);
@@ -23844,6 +25180,7 @@ values by @code{gal_statistics_mode} will have a value
of NaN.
@end deffn
@deffn Macro GAL_STATISTICS_SORTED_NOT
+@deffnx Macro GAL_STATISTICS_SORTED_INVALID
@deffnx Macro GAL_STATISTICS_SORTED_INCREASING
@deffnx Macro GAL_STATISTICS_SORTED_DECREASING
Macros used to identify if the dataset is sorted and increasing, sorted and
@@ -23858,8 +25195,8 @@ Macros used to identify if the regularity of the bins
when defining bins.
@cindex Number
@deftypefun {gal_data_t *} gal_statistics_number (gal_data_t @code{*input})
-Return a single-element @code{uint64} dataset containing the number of
-non-blank elements in @code{input}.
+Return a single-element dataset with type @code{size_t} which contains the
+number of non-blank elements in @code{input}.
@end deftypefun
@cindex Minimum
@@ -23977,8 +25314,10 @@ array[3]: value at the end of symmetricity.
@deftypefun {gal_data_t *} gal_statistics_mode_mirror_plots (gal_data_t
@code{*input}, gal_data_t @code{*value}, size_t @code{numbins}, int
@code{inplace}, double @code{*mirror_val})
Make a mirrored histogram and cumulative frequency plot (with
-@code{numbins}) with the mirror distribution of the @code{input} with a
-value at @code{value}.
+@code{numbins}) with the mirror distribution of the @code{input} having a
+value in @code{value}. If all the input elements are blank, or the mirror
+value is outside the range of the input, this function will return a
+@code{NULL} pointer.
The output is a list of data structures (see @ref{List of gal_data_t}): the
first is the bins with one bin at the mirror point, the second is the
@@ -23988,7 +25327,9 @@ plot (with a maximum of one).
@deftypefun int gal_statistics_is_sorted (gal_data_t @code{*input})
-Return the respective sort macro (see above) for the @code{input} dataset.
+Return the respective sort macro (see above) for the @code{input}
+dataset. If the dataset has zero elements, then
+@code{GAL_STATISTICS_SORTED_INVALID} is returned.
@end deftypefun
@deftypefun void gal_statistics_sort_increasing (gal_data_t @code{*input})
@@ -24004,6 +25345,12 @@ Remove all the blanks and sort the input dataset. If
@code{inplace} is
non-zero this will happen on the input dataset (and the output will be the
same as the input). However, if @code{inplace} is zero, this function will
allocate a new copy of the dataset that is sorted and has no blank values.
+
+If all the elements were blank, then the returned dataset's @code{size}
+will be zero. This is thus a good parameter to check after calling this
+function to see if there actually were any non-blank elements in the input
+or not and take the appropriate measure. This can help avoid strange bugs
+in later steps.
@end deftypefun
@deftypefun {gal_data_t *} gal_statistics_regular_bins (gal_data_t
@code{*input}, gal_data_t @code{*inrange}, size_t @code{numbins}, double
@code{onebinstart})
@@ -24083,8 +25430,8 @@ normalized CFP's maximum value is 1.
Apply @mymath{\sigma}-clipping on a given dataset and return a dataset that
contains the results. For a description of @mymath{\sigma}-clipping see
@ref{Sigma clipping}. @code{multip} is the multiple of the standard
-deviation (@mymath{\sigma} that is used to define outliers in each round of
-clipping).
+deviation (or @mymath{\sigma}, that is used to define outliers in each
+round of clipping).
The role of @code{param} is determined based on its value. If @code{param}
is larger than @code{1} (one), it must be an integer and will be
@@ -24099,6 +25446,10 @@ array[1]: Median.
array[2]: Mean.
array[3]: Standard deviation.
@end example
+
+If the @mymath{\sigma}-clipping doesn't converge or all input elements are
+blank, then this function will return NaN values for all the elements
+above.
@end deftypefun
@@ -24217,8 +25568,8 @@ applied on the dataset, then @code{num} dilations.
@deftypefun size_t gal_binary_connected_components (gal_data_t @code{*binary},
gal_data_t @code{**out}, int @code{connectivity})
-@cindex Connected components
@cindex Breadth first search
+@cindex Connected component labeling
Return the number of connected components in @code{binary} through the
breadth first search algorithm (finding all pixels belonging to one
component before going on to the next). Connection between two pixels is
@@ -24530,8 +25881,7 @@ main(void)
/* Read `img.fits' (HDU: 1) as a float32 array. */
- image=gal_fits_img_read_to_type(filename, hdu, GAL_TYPE_FLOAT32, -1,
- 0, 0);
+ image=gal_fits_img_read_to_type(filename, hdu, GAL_TYPE_FLOAT32, -1);
/* Use the allocated space as a single precision floating
@@ -24580,8 +25930,7 @@ main(void)
float *array;
size_t i, num, *dinc;
gal_data_t *input=gal_fits_img_read_to_type("input.fits", "1",
- GAL_TYPE_FLOAT32, -1,
- 0, 0);
+ GAL_TYPE_FLOAT32, -1);
/* To avoid the `void *' pointer and have `dinc'. */
array=input->array;
@@ -24721,8 +26070,7 @@ main(void)
/* Read the image into memory as a float32 data type. We are using
* `-1' for `minmapsize' to ensure that the image is read into
* memory. */
- p.image=gal_fits_img_read_to_type(filename, hdu, GAL_TYPE_FLOAT32,
- -1, 0, 0);
+ p.image=gal_fits_img_read_to_type(filename, hdu, GAL_TYPE_FLOAT32, -1);
/* Print some basic information before the actual contents: */
@@ -25611,13 +26959,17 @@ use all the great features that Gnuastro has already
defined for such
operations.
To make it easier to learn/apply the internal program infra-structure
-discussed in @ref{Mandatory source code files}, Gnuastro ships with a
-template program when using the @ref{Version controlled source}. It is not
-available in the Gnuastro tarball so it doesn't confuse people using the
-tarball. The @file{bin/TEMPLATE} directory in Gnuastro's clone contains the
-bare-minimum files necessary to define a new program and all the necessary
-files/functions are pre-defined there. You can take the following steps if
-you want to add a new program to Gnuastro:
+discussed in @ref{Mandatory source code files}, in the @ref{Version
+controlled source}, Gnuastro ships with a template program . This template
+program is not available in the Gnuastro tarball so it doesn't confuse
+people using the tarball. The @file{bin/TEMPLATE} directory in Gnuastro's
+Git repository contains the bare-minimum files necessary to define a new
+program and all the basic/necessary files/functions are pre-defined
+there.
+
+Below you can see a list of initial steps to take for customizing this
+template. We just assume that after cloning Gnuastro's history, you have
+already bootstrapped Gnuastro, if not, please see @ref{Bootstrapping}.
@enumerate
@item
@@ -25630,32 +26982,84 @@ $ cp -R bin/TEMPLATE bin/myprog
@end example
@item
+As with all source files in Gnuastro, all the files in template also have a
+copyright notice at their top. Open all the files and correct these
+notices: 1) The first line contains a single-line description of the
+program. 2) In the second line only the name or your program needs to be
+fixed and 3) Add your name and email as a ``Contributing author''. As your
+program grows, you will need to add new files, don't forget to add this
+notice in those new files too, just put your name and email under
+``Original author'' and correct the copyright years.
+
+@item
Open @file{configure.ac} in the top Gnuastro source. This file manages the
operations that are done when a user runs @file{./configure}. Going down
-the file, you will notice repetitive parts for each program. Copy one of
-those and correct the names of the copied program to your new program
-name. We follow alphabetic ordering here, so please place it
-correctly. There are multiple places where this has to be done, so be
+the file, you will notice repetitive parts for each program. You will
+notice that the program names follow an alphabetic ordering in each
+part. There is also a commented line/patch for the @file{TEMPLATE} program
+in each part. You can copy one line/patch (from the program above or below
+your desired name for example) and paste it in the proper place for your
+new program. Then correct the names of the copied program to your new
+program name. There are multiple places where this has to be done, so be
patient and go down to the bottom of the file. Ultimately add
@file{bin/myprog/Makefile} to @code{AC_CONFIG_FILES}, only here the
-ordering depends on the length of the name.
+ordering depends on the length of the name (it isn't alphabetical).
@item
Open @file{Makefile.am} in the top Gnuastro source. Similar to the previous
-step, add your new program similar to all the other programs.
+step, add your new program similar to all the other programs. Here there
+are only two places: 1) at the top where we define the conditionals (three
+lines per program), and 2) immediately under it as part of the value for
+@code{SUBDIRS}.
+
+@item
+Open @file{doc/Makefile.am} and similar to @file{Makefile.am} (above), add
+the proper entries for the man-page of your program to be created (here,
+the variable that keeps all the man-pages to be created is
+@code{dist_man_MANS}). Then scroll down and add a rule to build the
+man-page similar to the other existing rules (in alphabetical order). Don't
+forget to add a short one-line description here, it will be displayed on
+top of the man-page.
+
+@item
+Change @code{TEMPLATE.c} and @code{TEMPLATE.h} to @code{myprog.c} and
+@code{myprog.h} in the file names:
+
+@example
+$ cd bin/myprog
+$ mv TEMPLATE.c myprog.c
+$ mv TEMPLATE.h myprog.h
+@end example
@item
-Change @code{TEMPLATE} to @code{myprog} in the file names and contents of
-the files in the @file{bin/myprog/} directory.
+@cindex GNU Grep
+Correct all occurances of @code{TEMPLATE} in the input files to
+@code{myprog} (in short or long format). You can get a list of all
+occurances with the following command. If you use Emacs, it will be able to
+parse the Grep output and open the proper file and line automatically. So
+this step can be very easy.
+
+@example
+$ grep --color -nHi -e template *
+@end example
@item
-Run the following command to re-build the configuration and build system.
+Run the following commands to re-build the configuration and build system,
+and then to configure and build Gnuastro (which now includes your exciting
+new program).
@example
$ autoreconf -f
+$ ./configure
+$ make
@end example
-Your new program will be built the next time you run @command{./configure}
-and @command{make}. You can now start adding your special
-checks/processing.
+
+@item
+You are done! You can now start customizing your new program to do your
+special processing. When its complete, just don't forget to add checks
+also, so it can be tested at least once on a user's system with
+@command{make check}, see @ref{Test scripts}. Finally, if you would like to
+share it with all Gnuastro users, inform us so we merge it into Gnuastro's
+main history.
@end enumerate
@@ -26507,10 +27911,6 @@ coordinates.
(@file{astfits}, see @ref{Fits}) View and manipulate FITS file extensions
and header keywords.
-@item Statistics
-(@file{aststatistics}, see @ref{Statistics}) Get pixel statistics and
-save histogram and cumulative frequency plots.
-
@item MakeCatalog
(@file{astmkcatalog}, see @ref{MakeCatalog}) Make catalog of labeled image
(output of NoiseChisel). The catalogs are highly customizable and adding
@@ -26531,11 +27931,18 @@ over-sampled image.
that match with each other within a given aperture (may be an ellipse).
@item NoiseChisel
-(@file{astnoisechisel}, see @ref{NoiseChisel}) Detect and segment signal in
-noise. It uses a technique to detect very faint and diffuse, irregularly
-shaped signal in noise (galaxies in the sky), using thresholds that are
-below the Sky value, see @url{http://arxiv.org/abs/1505.01664,
-arXiv:1505.01664}.
+(@file{astnoisechisel}, see @ref{NoiseChisel}) Detect signal in noise. It
+uses a technique to detect very faint and diffuse, irregularly shaped
+signal in noise (galaxies in the sky), using thresholds that are below the
+Sky value, see @url{http://arxiv.org/abs/1505.01664, arXiv:1505.01664}.
+
+@item Segment
+(@file{astsegment}, see @ref{Segment}) Segment detected regions based on
+the structure of signal and the input dataset's noise properties.
+
+@item Statistics
+(@file{aststatistics}, see @ref{Statistics}) Statistical calculations on
+the input dataset (column in a table, image or datacube).
@item Table
(@file{asttable}, @ref{Table}) Convert FITS binary and ASCII tables into
diff --git a/doc/release-checklist.txt b/doc/release-checklist.txt
index 3cf287f..e6cb40a 100644
--- a/doc/release-checklist.txt
+++ b/doc/release-checklist.txt
@@ -183,6 +183,25 @@ Steps necessary to Package Gnuastro for Debian.
$ sudo apt-get update
$ sudo apt-get upgrade
+ - If this is the first time you are packaging on this system, you will
+ need to install the following programs:
+
+ $ sudo apt-get install devscripts pbuilder pristine-tar git quilt lintian
+ $ sudo pbuilder create
+ $ emacs ~/.devscripts
+
+ Add these two lines to the opened file:
+
+ DEBFULLNAME="Your name"
+ DEBEMAIL=your@email.address
+
+ Then add these lines to `~/.quiltrc':
+
+ QUILT_PATCHES=debian/patches
+ QUILT_NO_DIFF_INDEX=1
+ QUILT_NO_DIFF_TIMESTAMPS=1
+ QUILT_REFRESH_ARGS="-p ab"
+
A restart should help in making sure everything that has been updated is
being used.
diff --git a/lib/Makefile.am b/lib/Makefile.am
index 0ee4e9d..c978b4b 100644
--- a/lib/Makefile.am
+++ b/lib/Makefile.am
@@ -56,10 +56,11 @@ libgnuastro_la_SOURCES = arithmetic.c arithmetic-and.c
arithmetic-bitand.c \
arithmetic-bitxor.c arithmetic-divide.c arithmetic-eq.c arithmetic-ge.c \
arithmetic-gt.c arithmetic-le.c arithmetic-lt.c arithmetic-minus.c \
arithmetic-modulo.c arithmetic-multiply.c arithmetic-ne.c \
- arithmetic-or.c arithmetic-plus.c binary.c blank.c box.c checkset.c \
- convolve.c cosmology.c data.c fits.c git.c interpolate.c list.c match.c \
- options.c permutation.c polygon.c qsort.c dimension.c statistics.c \
- table.c tableintern.c threads.c tile.c timing.c txt.c type.c wcs.c
+ arithmetic-or.c arithmetic-plus.c array.c binary.c blank.c box.c \
+ checkset.c convolve.c cosmology.c data.c eps.c fits.c git.c \
+ interpolate.c jpeg.c label.c list.c match.c options.c pdf.c \
+ permutation.c polygon.c qsort.c dimension.c statistics.c table.c \
+ tableintern.c threads.c tiff.c tile.c timing.c txt.c type.c wcs.c
@@ -69,15 +70,17 @@ libgnuastro_la_SOURCES = arithmetic.c arithmetic-and.c
arithmetic-bitand.c \
# in the $(headersdir) directory. Some of the header files don't need to be
# installed.
headersdir=$(top_srcdir)/lib/gnuastro
-pkginclude_HEADERS = gnuastro/config.h $(headersdir)/arithmetic.h \
- $(headersdir)/binary.h $(headersdir)/blank.h $(headersdir)/box.h \
- $(headersdir)/convolve.h $(headersdir)/cosmology.h $(headersdir)/data.h \
- $(headersdir)/dimension.h $(headersdir)/fits.h $(headersdir)/git.h \
- $(headersdir)/interpolate.h $(headersdir)/list.h $(headersdir)/match.h \
- $(headersdir)/permutation.h $(headersdir)/polygon.h \
- $(headersdir)/qsort.h $(headersdir)/statistics.h $(headersdir)/table.h \
- $(headersdir)/threads.h $(headersdir)/tile.h $(headersdir)/txt.h \
- $(headersdir)/type.h $(headersdir)/wcs.h
+pkginclude_HEADERS = gnuastro/config.h $(headersdir)/arithmetic.h \
+ $(headersdir)/array.h $(headersdir)/binary.h $(headersdir)/blank.h \
+ $(headersdir)/box.h $(headersdir)/convolve.h $(headersdir)/cosmology.h \
+ $(headersdir)/data.h $(headersdir)/dimension.h $(headersdir)/eps.h \
+ $(headersdir)/fits.h $(headersdir)/git.h $(headersdir)/interpolate.h \
+ $(headersdir)/jpeg.h $(headersdir)/label.h $(headersdir)/list.h \
+ $(headersdir)/match.h $(headersdir)/pdf.h $(headersdir)/permutation.h \
+ $(headersdir)/polygon.h $(headersdir)/qsort.h $(headersdir)/statistics.h \
+ $(headersdir)/table.h $(headersdir)/threads.h $(headersdir)/tiff.h \
+ $(headersdir)/tile.h $(headersdir)/txt.h $(headersdir)/type.h \
+ $(headersdir)/wcs.h
@@ -102,6 +105,7 @@ EXTRA_DIST = gnuastro.pc.in $(headersdir)/README
$(internaldir)/README \
$(internaldir)/arithmetic-or.h $(internaldir)/arithmetic-plus.h \
$(internaldir)/checkset.h $(internaldir)/commonopts.h \
$(internaldir)/config.h.in $(internaldir)/fixedstringmacros.h \
+ $(internaldir)/kernel-2d.h $(internaldir)/kernel-3d.h \
$(internaldir)/options.h $(internaldir)/tableintern.h \
$(internaldir)/timing.h
diff --git a/lib/arithmetic-and.c b/lib/arithmetic-and.c
index bd00a61..3d144da 100644
--- a/lib/arithmetic-and.c
+++ b/lib/arithmetic-and.c
@@ -47,5 +47,5 @@ arithmetic_and(gal_data_t *l, gal_data_t *r, gal_data_t *o)
{
int checkblank=gal_arithmetic_binary_checkblank(l, r);
- BINARY_SET_LT( ARITHMETIC_BINARY_OUT_TYPE_UINT8, && );
+ BINARY_SET_LT( ARITHMETIC_BINARY_OUT_TYPE_INCR_SEP, && );
}
diff --git a/lib/arithmetic-or.c b/lib/arithmetic-or.c
index cd1dd41..716469e 100644
--- a/lib/arithmetic-or.c
+++ b/lib/arithmetic-or.c
@@ -47,5 +47,5 @@ arithmetic_or(gal_data_t *l, gal_data_t *r, gal_data_t *o)
{
int checkblank=gal_arithmetic_binary_checkblank(l, r);
- BINARY_SET_LT( ARITHMETIC_BINARY_OUT_TYPE_UINT8, || );
+ BINARY_SET_LT( ARITHMETIC_BINARY_OUT_TYPE_INCR_SEP, || );
}
diff --git a/lib/array.c b/lib/array.c
new file mode 100644
index 0000000..0aa708e
--- /dev/null
+++ b/lib/array.c
@@ -0,0 +1,174 @@
+/*********************************************************************
+array -- Functions for I/O on arrays (images or cubes)
+This is part of GNU Astronomy Utilities (Gnuastro) package.
+
+Original author:
+ Mohammad Akhlaghi <mohammad@akhlaghi.org>
+Contributing author(s):
+Copyright (C) 2018, Free Software Foundation, Inc.
+
+Gnuastro is free software: you can redistribute it and/or modify it
+under the terms of the GNU General Public License as published by the
+Free Software Foundation, either version 3 of the License, or (at your
+option) any later version.
+
+Gnuastro is distributed in the hope that it will be useful, but
+WITHOUT ANY WARRANTY; without even the implied warranty of
+MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the GNU
+General Public License for more details.
+
+You should have received a copy of the GNU General Public License
+along with Gnuastro. If not, see <http://www.gnu.org/licenses/>.
+**********************************************************************/
+#include <config.h>
+
+#include <stdio.h>
+#include <errno.h>
+#include <error.h>
+#include <stdlib.h>
+
+#include <gnuastro/txt.h>
+#include <gnuastro/fits.h>
+#include <gnuastro/jpeg.h>
+#include <gnuastro/tiff.h>
+#include <gnuastro/array.h>
+
+
+
+
+
+
+
+
+
+
+/*********************************************************************/
+/***************** High-level functions ****************/
+/*********************************************************************/
+int
+gal_array_name_recognized(char *name)
+{
+ if( gal_fits_name_is_fits(name) ) return 1;
+ else if ( gal_tiff_name_is_tiff(name) ) return 1;
+ else if ( gal_jpeg_name_is_jpeg(name) ) return 1;
+ else return 0;
+
+ /* Control should not get to here, but just to avoid compiler warnings,
+ we'll return a NULL. */
+ error(EXIT_FAILURE, 0, "%s: a bug! Please contact us at %s to solve the "
+ "problem. Control must not reach the end of this function", __func__,
+ PACKAGE_BUGREPORT);
+ return 0;
+}
+
+
+
+
+
+/* Read (all the possibly existing) color channels within each
+ extension/dir of the given file. */
+gal_data_t *
+gal_array_read(char *filename, char *extension, size_t minmapsize)
+{
+ size_t ext;
+
+ /* FITS */
+ if( gal_fits_name_is_fits(filename) )
+ return gal_fits_img_read(filename, extension, minmapsize);
+
+ /* TIFF */
+ else if ( gal_tiff_name_is_tiff(filename) )
+ {
+ ext=gal_tiff_dir_string_read(extension);
+ return gal_tiff_read(filename, ext, minmapsize);
+ }
+
+ /* JPEG */
+ else if ( gal_jpeg_name_is_jpeg(filename) )
+ return gal_jpeg_read(filename, minmapsize);
+
+ /* Default: plain text. */
+ else
+ return gal_txt_image_read(filename, minmapsize);
+
+ /* Control should not get to here, but just to avoid compiler warnings,
+ we'll return a NULL. */
+ error(EXIT_FAILURE, 0, "%s: a bug! Please contact us at %s to solve the "
+ "problem. Control must not reach the end of this function", __func__,
+ PACKAGE_BUGREPORT);
+ return NULL;
+}
+
+
+
+
+
+/* Read the contents of the given file/extension to a specific type. */
+gal_data_t *
+gal_array_read_to_type(char *filename, char *extension, uint8_t type,
+ size_t minmapsize)
+{
+ gal_data_t *out=NULL;
+ gal_data_t *next, *in=gal_array_read(filename, extension, minmapsize);
+
+ /* Go over all the channels. */
+ while(in)
+ {
+ next=in->next;
+ in->next=NULL;
+ gal_list_data_add(&out, gal_data_copy_to_new_type_free(in, type));
+ in=next;
+ }
+
+ /* Invert the reverse list and return. */
+ gal_list_data_reverse(&out);
+ return out;
+}
+
+
+
+
+
+/* Read the input array and make sure it is only one channel. */
+gal_data_t *
+gal_array_read_one_ch(char *filename, char *extension, size_t minmapsize)
+{
+ char *fname;
+ gal_data_t *out;
+ out=gal_array_read(filename, extension, minmapsize);
+
+ if(out->next)
+ {
+ if(extension)
+ {
+ if( asprintf(&fname, "%s (hdu %s)", filename, extension)<0 )
+ error(EXIT_FAILURE, 0, "%s: asprintf allocation error", __func__);
+ }
+ else
+ fname=filename;
+
+ error(EXIT_FAILURE, 0, "%s: contains %zu channels (it isn't "
+ "monochrome).\n\n"
+ "You can use Gnuastro's ConvertType program to separate the "
+ "(color) channels into separate extensions of a FITS file, with "
+ "a command like this:\n\n"
+ " $ astconvertt %s -h%s --output=sep-ch.fits",
+ fname, gal_list_data_number(out), filename, extension);
+ }
+
+ return out;
+}
+
+
+
+
+
+/* Read a single-channel dataset into a specific type. */
+gal_data_t *
+gal_array_read_one_ch_to_type(char *filename, char *extension, uint8_t type,
+ size_t minmapsize)
+{
+ gal_data_t *out=gal_array_read_one_ch(filename, extension, minmapsize);
+
+ return gal_data_copy_to_new_type_free(out, type);
+}
diff --git a/lib/blank.c b/lib/blank.c
index 98b69f3..11cbd6b 100644
--- a/lib/blank.c
+++ b/lib/blank.c
@@ -112,6 +112,57 @@ gal_blank_initialize(gal_data_t *input)
+
+/* Return 1 if the contents of the pointer (with the given type) is
+ blank. */
+int
+gal_blank_is(void *pointer, uint8_t type)
+{
+ switch(type)
+ {
+ /* Numeric types */
+ case GAL_TYPE_UINT8: return *(uint8_t *)pointer==GAL_BLANK_UINT8;
+ case GAL_TYPE_INT8: return *(int8_t *)pointer==GAL_BLANK_INT8;
+ case GAL_TYPE_UINT16: return *(uint16_t *)pointer==GAL_BLANK_UINT16;
+ case GAL_TYPE_INT16: return *(int16_t *)pointer==GAL_BLANK_INT16;
+ case GAL_TYPE_UINT32: return *(uint32_t *)pointer==GAL_BLANK_UINT32;
+ case GAL_TYPE_INT32: return *(int32_t *)pointer==GAL_BLANK_INT32;
+ case GAL_TYPE_UINT64: return *(uint64_t *)pointer==GAL_BLANK_UINT64;
+ case GAL_TYPE_INT64: return *(int64_t *)pointer==GAL_BLANK_INT64;
+ case GAL_TYPE_FLOAT32: return isnan( *(float *)(pointer) );
+ case GAL_TYPE_FLOAT64: return isnan( *(double *)(pointer) );
+
+ /* String. */
+ case GAL_TYPE_STRING: if(!strcmp(pointer,GAL_BLANK_STRING)) return 1;
+
+ /* Complex types */
+ case GAL_TYPE_COMPLEX32:
+ case GAL_TYPE_COMPLEX64:
+ error(EXIT_FAILURE, 0, "%s: complex types are not yet supported",
+ __func__);
+
+ /* Bit. */
+ case GAL_TYPE_BIT:
+ error(EXIT_FAILURE, 0, "%s: bit type datasets are not yet supported",
+ __func__);
+
+ default:
+ error(EXIT_FAILURE, 0, "%s: type value (%d) not recognized",
+ __func__, type);
+ }
+
+ /* Control should not reach here, so print an error if it does, then
+ return a 0 (just to avoid compiler warnings). */
+ error(EXIT_FAILURE, 0, "%s: a bug! Please contact us at %s to address the "
+ "problem. Control should not reach the end of this funciton",
+ __func__, PACKAGE_BUGREPORT);
+ return 0;
+}
+
+
+
+
+
/* Return 1 if the dataset has a blank value and zero if it doesn't. Before
checking the dataset, this function will look at its flags. If the
`GAL_DATA_FLAG_HASBLANK' or `GAL_DATA_FLAG_DONT_CHECK_ZERO' bits of
diff --git a/lib/data.c b/lib/data.c
index d3e85e1..9317b62 100644
--- a/lib/data.c
+++ b/lib/data.c
@@ -450,7 +450,7 @@ gal_data_free_contents(gal_data_t *data)
if(data->type==GAL_TYPE_STRING && data->array)
{
strarr=data->array;
- for(i=0;i<data->size;++i) free(strarr[i]);
+ for(i=0;i<data->size;++i) if(strarr[i]) free(strarr[i]);
}
/* Free the array. */
@@ -533,9 +533,13 @@ gal_data_array_calloc(size_t size)
out[i].type = GAL_TYPE_INVALID;
out[i].ndim = 0;
out[i].dsize = NULL;
+ out[i].size = 0;
+ out[i].mmapname = NULL;
+ out[i].minmapsize = -1;
out[i].nwcs = 0;
out[i].wcs = NULL;
- out[i].mmapname = NULL;
+ out[i].flag = 0;
+ out[i].status = 0;
out[i].next = NULL;
out[i].block = NULL;
out[i].name = out[i].unit = out[i].comment = NULL;
diff --git a/lib/eps.c b/lib/eps.c
new file mode 100644
index 0000000..7a530b4
--- /dev/null
+++ b/lib/eps.c
@@ -0,0 +1,438 @@
+/*********************************************************************
+eps -- functions to write EPS files.
+This is part of GNU Astronomy Utilities (Gnuastro) package.
+
+Original author:
+ Mohammad Akhlaghi <mohammad@akhlaghi.org>
+Contributing author(s):
+Copyright (C) 2015-2018, Free Software Foundation, Inc.
+
+Gnuastro is free software: you can redistribute it and/or modify it
+under the terms of the GNU General Public License as published by the
+Free Software Foundation, either version 3 of the License, or (at your
+option) any later version.
+
+Gnuastro is distributed in the hope that it will be useful, but
+WITHOUT ANY WARRANTY; without even the implied warranty of
+MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the GNU
+General Public License for more details.
+
+You should have received a copy of the GNU General Public License
+along with Gnuastro. If not, see <http://www.gnu.org/licenses/>.
+**********************************************************************/
+#include <config.h>
+
+#include <stdio.h>
+#include <errno.h>
+#include <error.h>
+#include <stdlib.h>
+#include <string.h>
+
+#include <gnuastro/eps.h>
+#include <gnuastro/list.h>
+
+#include <gnuastro-internal/timing.h>
+#include <gnuastro-internal/checkset.h>
+
+
+
+
+
+
+/*************************************************************
+ ************** Acceptable EPS names ***************
+ *************************************************************/
+int
+gal_eps_name_is_eps(char *name)
+{
+ size_t len;
+ len=strlen(name);
+ if ( ( len>=3 && strcmp(&name[len-3], "eps") == 0 )
+ || ( len>=3 && strcmp(&name[len-3], "EPS") == 0 )
+ || ( len>=4 && strcmp(&name[len-4], "epsf") == 0 )
+ || ( len>=4 && strcmp(&name[len-4], "epsi") == 0 ) )
+ return 1;
+ else
+ return 0;
+}
+
+
+
+
+
+int
+gal_eps_suffix_is_eps(char *name)
+{
+ if (strcmp(name, "eps") == 0 || strcmp(name, ".eps") == 0
+ || strcmp(name, "EPS") == 0 || strcmp(name, ".EPS") == 0
+ || strcmp(name, "epsf") == 0 || strcmp(name, ".epsf") == 0
+ || strcmp(name, "epsi") == 0 || strcmp(name, ".epsi") == 0)
+ return 1;
+ else
+ return 0;
+}
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+/*************************************************************
+ ************** Write an EPS image **************
+ *************************************************************/
+static int
+eps_is_binary(gal_data_t *in, uint8_t *bitone)
+{
+ gal_data_t *channel;
+ uint8_t second_set=0;
+ unsigned char *i, *fi, first=0, second=0;
+
+ /* Go through all the channels. */
+ for(channel=in; channel!=NULL; channel=channel->next)
+ {
+ /* Go through all the values and see if there is more than two values
+ in the array. */
+ fi = (i=channel->array) + channel->size;
+ first=*i;
+ do
+ if(*i!=first)
+ {
+ if(second_set)
+ { if(*i!=second) break; }
+ else
+ { second=*i; second_set=1; }
+ }
+ while(++i<fi);
+
+ /* If we didn't get to the end of the array, then we have a
+ multi-valued (not binary) image. */
+ if(i!=fi)
+ return 0;
+ }
+
+ /* If we get to this point, then all the channels were binary, so return
+ success. */
+ *bitone = first>second ? first : second;
+ return 1;
+}
+
+
+
+
+
+/* Convert the channels into into a 0 and 1 bit stream. This function is
+ only called when the image is binary (has only two values). NOTE: each
+ row has to have an integer number of bytes, so when the number of pixels
+ in a row is not a multiple of 8, we'll add one. */
+static gal_data_t *
+eps_convert_to_bitstream(gal_data_t *in, size_t *numbytes, uint8_t bitone)
+{
+ size_t i, j, k, bytesinrow;
+ gal_data_t *channel, *out=NULL;
+ unsigned char *bits, byte, curbit, *arr;
+ size_t s0=in->dsize[0], s1=in->dsize[1];
+
+ /* Find the size values and allocate the array. */
+ if( s1 % 8 ) bytesinrow = s1/8 + 1;
+ else bytesinrow = s1/8;
+ *numbytes = bytesinrow*s0;
+
+ /* Go over all the channels. */
+ for(channel=in; channel!=NULL; channel=channel->next)
+ {
+ /* Allocate the array. Note that we currently don't have an
+ allocation system for bits, so we'll allocate space in bytes, then
+ convert */
+ gal_list_data_add_alloc(&out, NULL, GAL_TYPE_UINT8, 1, numbytes,
+ NULL, 0, -1, NULL, NULL, NULL);
+ out->type=GAL_TYPE_BIT;
+ bits=out->array;
+
+ /* Put the values in. */
+ arr=channel->array;
+ for(i=0;i<s0;++i) /* i*s0+j is the byte, not bit position. */
+ for(j=0;j<bytesinrow;++j)
+ {
+ /* Set the 8 bits to zero. */
+ byte=0;
+
+ /* Current bit position. */
+ curbit=0x80;
+
+ /* Write the next 8 values as bits. */
+ for(k=0;k<8;++k)
+ {
+ if( j*8+k < s1 )
+ {
+ if(arr[i*s1+j*8+k]==bitone)
+ byte |= curbit;
+ curbit >>= 1;
+ }
+ else break;
+ }
+
+ /* Write the byte into the array. */
+ bits[i*bytesinrow+j]=byte;
+ }
+ }
+
+ /* Reverse the list and return it. */
+ gal_list_data_reverse(&out);
+ return out;
+}
+
+
+
+
+
+static void
+eps_write_hex(gal_data_t *write, FILE *fp, size_t numbytes)
+{
+ unsigned char *arr;
+ gal_data_t *channel;
+ size_t i=0, j, elem_for_newline=35;
+
+ for(channel=write; channel!=NULL; channel=channel->next)
+ {
+ if(channel->status) /* A blank channel has status==1. */
+ fprintf(fp, "{<00>} %% Channel %zu is blank\n", i);
+ else
+ {
+ arr=channel->array;
+ fprintf(fp, "{<");
+ for(j=0;j<numbytes;++j)
+ {
+ fprintf(fp, "%02X", arr[j]);
+ if(j%elem_for_newline==0) fprintf(fp, "\n");
+ }
+ fprintf(fp, ">}\n");
+ }
+ ++i;
+ }
+}
+
+
+
+
+
+static void
+eps_write_ascii85(gal_data_t *write, FILE *fp, size_t numbytes)
+{
+ unsigned char *arr;
+ gal_data_t *channel;
+ uint32_t anint, base;
+ size_t i=0, j, k, elem_for_newline=15; /* 15*5=75 */
+
+ for(channel=write; channel!=NULL; channel=channel->next)
+ {
+ if(channel->status)
+ fprintf(fp, "{<00>} %% Channel %zu is blank\n", i);
+ else
+ {
+ arr=channel->array;
+ fprintf(fp, "{<~");
+ for(j=0;j<numbytes;j+=4)
+ {
+ /* This is the last four bytes */
+ if(numbytes-j<4)
+ {
+ anint=arr[j]*256*256*256;
+ if(numbytes-j>1) anint+=arr[j+1]*256*256;
+ if(numbytes-j==3) anint+=arr[j+2]*256;
+ }
+ else
+ anint=( arr[j]*256*256*256 + arr[j+1]*256*256
+ + arr[j+2]*256 + arr[j+3] );
+
+ /* If all four bytes are zero, then just print `z'. */
+ if(anint==0) fprintf(fp, "z");
+ else
+ {
+ /* To check, just change the fprintf below to printf:
+ printf("\n\n");
+ printf("%u %u %u %u\n", in[i], in[i+1],
+ in[i+2], in[i+3]);
+ */
+ base=85*85*85*85;
+ /* Do the ASCII85 encoding: */
+ for(k=0;k<5;++k)
+ {
+ fprintf(fp, "%c", anint/base+33);
+ anint%=base;
+ base/=85;
+ }
+ }
+ /* Go to the next line if on the right place: */
+ if(j%elem_for_newline==0) fprintf(fp, "\n");
+ }
+ fprintf(fp, "~>}\n");
+ }
+ ++i;
+ }
+}
+
+
+
+
+
+static void
+eps_write_image(gal_data_t *in, FILE *fp, int hex)
+{
+ int bpc=8;
+ uint8_t bitone;
+ gal_data_t *write;
+ size_t i, numbytes, *dsize=in->dsize;
+ size_t numch=gal_list_data_number(in);
+
+ /* Set the number of bits per component. */
+ if( numch==1 && eps_is_binary(in, &bitone) )
+ {
+ bpc=1;
+ write=eps_convert_to_bitstream(in, &numbytes, bitone);
+ }
+ else
+ {
+ write=in;
+ numbytes=in->size;
+ }
+
+ /* Write the basic meta data. */
+ switch(numch)
+ {
+ case 1: fprintf(fp, "/DeviceGray setcolorspace\n"); break;
+ case 3: fprintf(fp, "/DeviceRGB setcolorspace\n"); break;
+ case 4: fprintf(fp, "/DeviceCMYK setcolorspace\n"); break;
+ default:
+ error(EXIT_FAILURE, 0, "%s: a bug! The number of channels (%zu) is "
+ "not 1, 3 or 4. Please contact us so we can find the issue and "
+ "fix it", __func__, numch);
+ }
+ fprintf(fp, "<<\n");
+ fprintf(fp, " /ImageType 1\n");
+ fprintf(fp, " /Width %zu\n", dsize[1]);
+ fprintf(fp, " /Height %zu\n", dsize[0]);
+ fprintf(fp, " /ImageMatrix [ %zu 0 0 %zu 0 0 ]\n", dsize[1], dsize[0]);
+ fprintf(fp, " /MultipleDataSources true\n");
+ fprintf(fp, " /BitsPerComponent %d\n", bpc);
+ fprintf(fp, " /Decode[");
+ for(i=0;i<numch;++i) {fprintf(fp, " 0 1");} fprintf(fp, " ]\n");
+ fprintf(fp, " /Interpolate false\n");
+ fprintf(fp, " /DataSource [\n");
+
+ /* Based on the encoding, write the contents of the image. */
+ if(hex)
+ eps_write_hex(write, fp, numbytes);
+ else
+ eps_write_ascii85(write, fp, numbytes);
+
+ /* Finish the file. */
+ fprintf(fp, " ]\n");
+ fprintf(fp, ">>\n");
+ fprintf(fp, "image\n\n");
+
+ /* Clean up. */
+ if(write!=in)
+ gal_list_data_free(write);
+}
+
+
+
+
+
+void
+gal_eps_to_pt(float widthincm, size_t *dsize, size_t *w_h_in_pt)
+{
+ w_h_in_pt[0] = widthincm*72.0f/2.54f;
+ w_h_in_pt[1] = (float)( dsize[0] * w_h_in_pt[0] )/(float)(dsize[1]);
+}
+
+
+
+
+
+void
+gal_eps_write(gal_data_t *in, char *filename, float widthincm,
+ uint32_t borderwidth, int hex, int forpdf)
+{
+ FILE *fp;
+ float hbw;
+ time_t rawtime;
+ size_t numch=gal_list_data_number(in);
+ size_t w_h_in_pt[2], *dsize=in->dsize;
+
+ /* Sanity checks. */
+ if(numch==2 || numch>4)
+ error(EXIT_FAILURE, 0, "%s: only 1, 3, and 4 color channels are "
+ "acceptable, input is a list of %zu data sets", __func__, numch);
+ if(in->type!=GAL_TYPE_UINT8)
+ error(EXIT_FAILURE, 0, "%s: input has a `%s' type, but JPEG images can "
+ "only have a `uint8' type", __func__, gal_type_name(in->type, 1));
+
+
+ /* Read the time to write in the output. */
+ time(&rawtime);
+
+
+ /* Find the bounding box */
+ hbw=(float)borderwidth/2.0;
+ gal_eps_to_pt(widthincm, dsize, w_h_in_pt);
+
+
+ /* Open the output file and write the top comments. */
+ errno=0;
+ fp=fopen(filename, "w");
+ if(fp==NULL)
+ error(EXIT_FAILURE, errno, "%s", filename);
+ fprintf(fp, "%%!PS-Adobe-3.0 EPSF-3.0\n");
+ fprintf(fp, "%%%%BoundingBox: 0 0 %zu %zu\n", w_h_in_pt[0]+2*borderwidth,
+ w_h_in_pt[1]+2*borderwidth);
+ fprintf(fp, "%%%%Creator: %s\n", PACKAGE_STRING);
+ fprintf(fp, "%%%%CreationDate: %s", ctime(&rawtime));
+ fprintf(fp, "%%%%LanuageLevel: 3\n");
+ fprintf(fp, "%%%%EndComments\n\n");
+ if(forpdf==0)
+ fprintf(fp, "gsave\n\n");
+
+
+ /* Commands to draw the border: */
+ if(borderwidth)
+ {
+ fprintf(fp, "%% Draw the border:\n");
+ fprintf(fp, "0 setgray\n");
+ fprintf(fp, "%d setlinewidth\n", borderwidth);
+ fprintf(fp, "%.1f %.1f moveto\n", hbw, hbw);
+ fprintf(fp, "0 %zu rlineto\n", w_h_in_pt[1]+borderwidth);
+ fprintf(fp, "%zu 0 rlineto\n", w_h_in_pt[0]+borderwidth);
+ fprintf(fp, "0 -%zu rlineto\n", w_h_in_pt[1]+borderwidth);
+ fprintf(fp, "closepath\n");
+ fprintf(fp, "stroke\n\n");
+ }
+
+
+ /* Write the image: */
+ fprintf(fp, "%% Draw the image:\n");
+ fprintf(fp, "%d %d translate\n", borderwidth, borderwidth);
+ fprintf(fp, "%zu %zu scale\n", w_h_in_pt[0], w_h_in_pt[1]);
+ eps_write_image(in, fp, hex);
+
+
+ /* Ending of the EPS file: */
+ if(forpdf) fprintf(fp, "showpage\n");
+ else fprintf(fp, "grestore\n");
+ fprintf(fp, "%%%%EOF");
+ fclose(fp);
+}
diff --git a/lib/fits.c b/lib/fits.c
index aadd92e..8ef4c7e 100644
--- a/lib/fits.c
+++ b/lib/fits.c
@@ -966,12 +966,12 @@ gal_fits_key_read_from_ptr(fitsfile *fptr, gal_data_t
*keysll,
switch(tmp->type)
{
case GAL_TYPE_STRING:
- errno=0;
tmp->array=strarray=( tmp->array
? tmp->array
: gal_data_malloc_array(tmp->type, 1,
__func__,
"tmp->array") );
+ errno=0;
valueptr=strarray[0]=malloc(FLEN_VALUE * sizeof *strarray[0]);
if(strarray[0]==NULL)
error(EXIT_FAILURE, errno, "%s: %zu bytes for strarray[0]",
@@ -1030,7 +1030,6 @@ gal_fits_key_read_from_ptr(fitsfile *fptr, gal_data_t
*keysll,
/* Strings need to be cleaned (CFITSIO puts `'' around them with
some (possiblly) extra space on the two ends of the string. */
-
}
}
@@ -1506,8 +1505,7 @@ gal_fits_img_info(fitsfile *fptr, int *type, size_t
*ndim, size_t **dsize,
/* Read a FITS image HDU into a Gnuastro data structure. */
gal_data_t *
-gal_fits_img_read(char *filename, char *hdu, size_t minmapsize,
- size_t hstartwcs, size_t hendwcs)
+gal_fits_img_read(char *filename, char *hdu, size_t minmapsize)
{
void *blank;
long *fpixel;
@@ -1552,6 +1550,8 @@ gal_fits_img_read(char *filename, char *hdu, size_t
minmapsize,
img=gal_data_alloc(NULL, type, ndim, dsize, NULL, 0, minmapsize,
name, unit, NULL);
blank=gal_blank_alloc_write(type);
+ if(name) free(name);
+ if(unit) free(unit);
free(dsize);
@@ -1563,10 +1563,6 @@ gal_fits_img_read(char *filename, char *hdu, size_t
minmapsize,
free(blank);
- /* Read the WCS structure (if the FITS file has any). */
- img->wcs=gal_wcs_read_fitsptr(fptr, hstartwcs, hendwcs, &img->nwcs);
-
-
/* Close the input FITS file. */
fits_close_file(fptr, &status);
gal_fits_io_error(status, NULL);
@@ -1585,13 +1581,12 @@ gal_fits_img_read(char *filename, char *hdu, size_t
minmapsize,
used to convert the input file to the desired type. */
gal_data_t *
gal_fits_img_read_to_type(char *inputname, char *hdu, uint8_t type,
- size_t minmapsize, size_t hstartwcs,
- size_t hendwcs)
+ size_t minmapsize)
{
gal_data_t *in, *converted;
/* Read the specified input image HDU. */
- in=gal_fits_img_read(inputname, hdu, minmapsize, hstartwcs, hendwcs);
+ in=gal_fits_img_read(inputname, hdu, minmapsize);
/* If the input had another type, convert it to float. */
if(in->type!=type)
@@ -1620,7 +1615,7 @@ gal_fits_img_read_kernel(char *filename, char *hdu,
size_t minmapsize)
/* Read the image as a float and if it has a WCS structure, free it. */
kernel=gal_fits_img_read_to_type(filename, hdu, GAL_TYPE_FLOAT32,
- minmapsize, 0, 0);
+ minmapsize);
if(kernel->wcs) { wcsfree(kernel->wcs); kernel->wcs=NULL; }
/* Check if the size along each dimension of the kernel is an odd
diff --git a/lib/gnuastro-internal/commonopts.h
b/lib/gnuastro-internal/commonopts.h
index 1e41a03..300f498 100644
--- a/lib/gnuastro-internal/commonopts.h
+++ b/lib/gnuastro-internal/commonopts.h
@@ -212,7 +212,7 @@ struct argp_option gal_commonopts_options[] =
GAL_OPTIONS_KEY_OUTPUT,
"STR",
0,
- "Output name.",
+ "Output file name.",
GAL_OPTIONS_GROUP_OUTPUT,
&cp->output,
GAL_TYPE_STRING,
diff --git a/lib/gnuastro-internal/kernel-2d.h
b/lib/gnuastro-internal/kernel-2d.h
new file mode 100644
index 0000000..ccc2d49
--- /dev/null
+++ b/lib/gnuastro-internal/kernel-2d.h
@@ -0,0 +1,129 @@
+/*********************************************************************
+The default 2D kernel to be used in NoiseChisel and Segment.
+This is part of GNU Astronomy Utilities (Gnuastro) package.
+
+Original author:
+ Mohammad Akhlaghi <mohammad@akhlaghi.org>
+Contributing author(s):
+Copyright (C) 2018, Free Software Foundation, Inc.
+
+Gnuastro is free software: you can redistribute it and/or modify it
+under the terms of the GNU General Public License as published by the
+Free Software Foundation, either version 3 of the License, or (at your
+option) any later version.
+
+Gnuastro is distributed in the hope that it will be useful, but
+WITHOUT ANY WARRANTY; without even the implied warranty of
+MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the GNU
+General Public License for more details.
+
+You should have received a copy of the GNU General Public License
+along with Gnuastro. If not, see <http://www.gnu.org/licenses/>.
+**********************************************************************/
+#ifndef __GAL_KERNEL2D_H__
+#define __GAL_KERNEL2D_H__
+
+
+/* How to produce this kernel
+ ==========================
+
+ Below, the steps necessary to easily create the C contents of this file
+ are described. The first step can be modified to change the default
+ kernel properties and put the new contents into this file.
+
+ Make the kernel
+ ---------------
+
+ We'll first make the kernel using MakeProfiles, then crop the outer
+ layers that are zero. Put these lines into a script and run it.
+
+ set -o errexit # Stop if a program returns false.
+ export GSL_RNG_TYPE=ranlxs2
+ export GSL_RNG_SEED=1
+ astmkprof --kernel=gaussian,2,5 --oversample=1 --envseed -ok2.fits
+ astcrop k2.fits --section=2:*-1,2:*-1 --zeroisnotblank \
+ --mode=img --output=fwhm2.fits
+ rm k2.fits
+
+ Convert it to C code
+ --------------------
+
+ We'll use this tiny C program to convert the FITS file into the two
+ important C variables:
+
+ #include <stdio.h>
+ #include <stdlib.h>
+ #include <gnuastro/fits.h>
+
+ int
+ main(void)
+ {
+ size_t i;
+ float *arr;
+ gal_data_t *img=gal_fits_img_read_to_type("fwhm2.fits", "1",
+ GAL_TYPE_FLOAT32, -1);
+
+ arr=img->array;
+
+ printf("size_t kernel_2d_dsize[2]={%zu, %zu};\n",
+ img->dsize[0], img->dsize[1]);
+ printf("float kernel_2d[%zu]={", img->size);
+ for(i=0;i<img->size;++i)
+ {
+ if(i>0)
+ {
+ if(i % img->dsize[1] == 0 ) printf("\n\n");
+ }
+
+ // We cannot use `\b' here, since we are writing directly
+ // to the command-line, so we'll first write the number,
+ // then decide if any subsequent character (a comma)
+ // should be written.
+ printf("%.7g", arr[i]);
+
+ // The last element doesn't need a comma. In each line,
+ // the last character must not be a space, but for easy
+ // readability, the elements in between need a space.
+ if( i!=(img->size-1) )
+ printf("%s", ((i+1)%img->dsize[1]) ? ", " : ",");
+ }
+ printf("};\n");
+
+ gal_data_free(img);
+ return EXIT_SUCCESS;
+ }
+
+ Run the C program
+ -----------------
+
+ We can now compile and run that C program and put the outputs in
+ `ready.c'. Once its created, copy the contents of `ready.c' after these
+ comments.
+
+ $ astbuildprog -q prepare.c > ready.c
+ */
+
+size_t kernel_2d_dsize[2]={11, 11};
+float kernel_2d[121]={0, 0, 0, 0, 0, 2.570758e-08, 0, 0, 0, 0, 0,
+
+0, 0, 2.981546e-08, 7.249833e-07, 4.468747e-06, 8.409227e-06, 4.554846e-06,
7.034199e-07, 3.002102e-08, 0, 0,
+
+0, 3.054498e-08, 2.614154e-06, 5.891601e-05, 0.0003810036, 0.000708165,
0.0003842406, 5.963722e-05, 2.618934e-06, 2.990584e-08, 0,
+
+0, 7.199899e-07, 5.801019e-05, 0.001365485, 0.009023659, 0.01638159,
0.008892864, 0.001345278, 5.920425e-05, 6.984741e-07, 0,
+
+0, 4.584869e-06, 0.0003830431, 0.008917402, 0.05743425, 0.1061489, 0.05746412,
0.008902563, 0.0003849257, 4.448404e-06, 0,
+
+2.572769e-08, 8.414041e-06, 0.0007008284, 0.0164456, 0.1055995, 0.19753,
0.1061855, 0.01653461, 0.0007141303, 8.41643e-06, 2.550312e-08,
+
+0, 4.582525e-06, 0.0003775396, 0.00898499, 0.05741534, 0.1062144, 0.05700329,
0.008838926, 0.0003822096, 4.543726e-06, 0,
+
+0, 6.883925e-07, 6.09077e-05, 0.001339333, 0.008817007, 0.01636454,
0.008995386, 0.001407854, 6.004799e-05, 7.203602e-07, 0,
+
+0, 3.095966e-08, 2.575403e-06, 5.89859e-05, 0.0003804447, 0.0007091904,
0.0003810006, 5.903253e-05, 2.575202e-06, 2.934356e-08, 0,
+
+0, 0, 3.040937e-08, 7.018197e-07, 4.543086e-06, 8.296753e-06, 4.434901e-06,
6.659026e-07, 3.066215e-08, 0, 0,
+
+0, 0, 0, 0, 0, 2.603901e-08, 0, 0, 0, 0, 0};
+
+#endif
diff --git a/lib/gnuastro-internal/kernel-3d.h
b/lib/gnuastro-internal/kernel-3d.h
new file mode 100644
index 0000000..30ca97e
--- /dev/null
+++ b/lib/gnuastro-internal/kernel-3d.h
@@ -0,0 +1,138 @@
+/*********************************************************************
+The default 3D kernel to be used in NoiseChisel and Segment.
+This is part of GNU Astronomy Utilities (Gnuastro) package.
+
+Original author:
+ Mohammad Akhlaghi <mohammad@akhlaghi.org>
+Contributing author(s):
+Copyright (C) 2018, Free Software Foundation, Inc.
+
+Gnuastro is free software: you can redistribute it and/or modify it
+under the terms of the GNU General Public License as published by the
+Free Software Foundation, either version 3 of the License, or (at your
+option) any later version.
+
+Gnuastro is distributed in the hope that it will be useful, but
+WITHOUT ANY WARRANTY; without even the implied warranty of
+MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the GNU
+General Public License for more details.
+
+You should have received a copy of the GNU General Public License
+along with Gnuastro. If not, see <http://www.gnu.org/licenses/>.
+**********************************************************************/
+#ifndef __GAL_KERNEL3D_H__
+#define __GAL_KERNEL3D_H__
+
+/* How to build this kernel.
+
+ Run MakeProfiles with the following commands:
+
+ $ export GSL_RNG_SEED=1
+ $ export GSL_RNG_TYPE=ranlxs2
+ $ astmkprof --kernel=gaussian-3d,1.5,5,0.5 --oversample=1 --envseed
+
+ The resulting fits file is converted to plain text with the this C
+ program to generate the `kernel-3d.h' header file which is then put
+ under these comments.
+
+ #include <stdio.h>
+ #include <stdlib.h>
+ #include <gnuastro/fits.h>
+
+ int
+ main(void)
+ {
+ size_t i;
+ float *arr;
+ gal_data_t *img=gal_fits_img_read_to_type("kernel.fits", "1",
+ GAL_TYPE_FLOAT32, -1);
+
+ arr=img->array;
+
+ printf("size_t kernel_3d_dsize[3]={%zu, %zu, %zu};\n",
+ img->dsize[0], img->dsize[1], img->dsize[2]);
+ printf("float kernel_3d[%zu]={", img->size);
+ for(i=0;i<img->size;++i)
+ {
+ if(i>0)
+ {
+ if(i % img->dsize[2] == 0 ) printf("\n");
+ if(i % (img->dsize[2]*img->dsize[1]) == 0 ) printf("\n");
+ }
+
+ // We cannot use `\b' here, since we are writing directly
+ // to the command-line, so we'll first write the number,
+ // then decide if any subsequent character (a comma)
+ // should be written.
+ printf("%.7g", arr[i]);
+
+ // The last element doesn't need a comma. In each line,
+ // the last character must not be a space, but for easy
+ // readability, the elements in between need a space.
+ if( i!=(img->size-1) )
+ printf("%s", ((i+1)%img->dsize[2]) ? ", " : ",");
+ }
+ printf("};\n");
+
+ return EXIT_SUCCESS;
+ }
+
+ Assuming this C program is in a file named `kernel.c', it can be
+ compiled, run and saved into `kernel-3d.h' with the command below:
+
+ $ astbuildprog -q kernel.c > kernel-3d.h
+
+ */
+
+size_t kernel_3d_dsize[3]={5, 9, 9};
+float kernel_3d[405]={0, 0, 0, 0, 0, 0, 0, 0, 0,
+0, 0, 0, 0, 0, 0, 0, 0, 0,
+0, 0, 0, 0, 0, 0, 0, 0, 0,
+0, 0, 0, 0, 0, 0, 0, 0, 0,
+0, 0, 0, 0, 0, 0, 0, 0, 0,
+0, 0, 0, 0, 0, 0, 0, 0, 0,
+0, 0, 0, 0, 0, 0, 0, 0, 0,
+0, 0, 0, 0, 0, 0, 0, 0, 0,
+0, 0, 0, 0, 0, 0, 0, 0, 0,
+
+0, 0, 0, 0, 0, 0, 0, 0, 0,
+0, 0, 0, 5.293481e-07, 1.391654e-06, 5.283476e-07, 0, 0, 0,
+0, 0, 5.023404e-06, 0.0001093707, 0.0003028791, 0.0001103754, 5.206822e-06, 0,
0,
+0, 5.27038e-07, 0.0001140605, 0.002517939, 0.006957374, 0.002468025,
0.0001131454, 5.31773e-07, 0,
+0, 1.481738e-06, 0.0002994444, 0.006708808, 0.01894199, 0.006776441,
0.0003116253, 1.493134e-06, 0,
+0, 5.293459e-07, 0.0001081432, 0.002473192, 0.006704316, 0.002578867,
0.0001105525, 4.866729e-07, 0,
+0, 0, 4.985141e-06, 0.000110672, 0.0002885369, 0.0001184523, 4.688276e-06, 0,
0,
+0, 0, 0, 5.610833e-07, 1.336052e-06, 5.216995e-07, 0, 0, 0,
+0, 0, 0, 0, 0, 0, 0, 0, 0,
+
+0, 0, 0, 0, 0, 0, 0, 0, 0,
+0, 0, 3.462786e-07, 7.853481e-06, 2.153742e-05, 7.564613e-06, 3.611804e-07, 0,
0,
+0, 3.52808e-07, 7.161698e-05, 0.001676089, 0.004618002, 0.001656355,
7.414426e-05, 3.43867e-07, 0,
+0, 7.990315e-06, 0.001704389, 0.03857445, 0.1031222, 0.03779064, 0.001703231,
7.975499e-06, 0,
+0, 2.231775e-05, 0.00473238, 0.1043504, 0.2858114, 0.10241, 0.004618072,
2.193002e-05, 0,
+0, 7.621142e-06, 0.001647367, 0.03755038, 0.1036048, 0.03796006, 0.001672143,
7.758124e-06, 0,
+0, 3.68421e-07, 7.659229e-05, 0.001695716, 0.004599371, 0.001670764,
7.450273e-05, 3.481919e-07, 0,
+0, 0, 3.521386e-07, 7.924394e-06, 2.207155e-05, 7.889852e-06, 3.53245e-07, 0,
0,
+0, 0, 0, 0, 0, 0, 0, 0, 0,
+
+0, 0, 0, 0, 0, 0, 0, 0, 0,
+0, 0, 0, 5.182329e-07, 1.397928e-06, 5.17901e-07, 0, 0, 0,
+0, 0, 4.812169e-06, 0.0001135803, 0.0003043477, 0.000109481, 4.974718e-06, 0,
0,
+0, 5.377616e-07, 0.0001073098, 0.002424047, 0.006817353, 0.002492242,
0.0001147361, 5.33439e-07, 0,
+0, 1.44089e-06, 0.0002933802, 0.006909565, 0.01873765, 0.006651767,
0.0003042257, 1.366647e-06, 0,
+0, 5.166781e-07, 0.0001119049, 0.002454791, 0.006940499, 0.002578837,
0.0001061706, 5.329928e-07, 0,
+0, 0, 4.656712e-06, 0.0001124511, 0.0003010639, 0.0001131462, 4.786908e-06, 0,
0,
+0, 0, 0, 5.303103e-07, 1.363214e-06, 5.122773e-07, 0, 0, 0,
+0, 0, 0, 0, 0, 0, 0, 0, 0,
+
+0, 0, 0, 0, 0, 0, 0, 0, 0,
+0, 0, 0, 0, 0, 0, 0, 0, 0,
+0, 0, 0, 0, 0, 0, 0, 0, 0,
+0, 0, 0, 0, 0, 0, 0, 0, 0,
+0, 0, 0, 0, 0, 0, 0, 0, 0,
+0, 0, 0, 0, 0, 0, 0, 0, 0,
+0, 0, 0, 0, 0, 0, 0, 0, 0,
+0, 0, 0, 0, 0, 0, 0, 0, 0,
+0, 0, 0, 0, 0, 0, 0, 0, 0};
+
+#endif
diff --git a/lib/gnuastro/array.h b/lib/gnuastro/array.h
new file mode 100644
index 0000000..aaff1fd
--- /dev/null
+++ b/lib/gnuastro/array.h
@@ -0,0 +1,75 @@
+/*********************************************************************
+array - Functions to read/write arrays from/to files.
+This is part of GNU Astronomy Utilities (Gnuastro) package.
+
+Original author:
+ Mohammad Akhlaghi <mohammad@akhlaghi.org>
+Contributing author(s):
+Copyright (C) 2018, Free Software Foundation, Inc.
+
+Gnuastro is free software: you can redistribute it and/or modify it
+under the terms of the GNU General Public License as published by the
+Free Software Foundation, either version 3 of the License, or (at your
+option) any later version.
+
+Gnuastro is distributed in the hope that it will be useful, but
+WITHOUT ANY WARRANTY; without even the implied warranty of
+MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the GNU
+General Public License for more details.
+
+You should have received a copy of the GNU General Public License
+along with Gnuastro. If not, see <http://www.gnu.org/licenses/>.
+**********************************************************************/
+#ifndef __GAL_ARRAY_H__
+#define __GAL_ARRAY_H__
+
+/* Include other headers if necessary here. Note that other header files
+ must be included before the C++ preparations below */
+#include <gnuastro/data.h>
+
+
+/* C++ Preparations */
+#undef __BEGIN_C_DECLS
+#undef __END_C_DECLS
+#ifdef __cplusplus
+# define __BEGIN_C_DECLS extern "C" {
+# define __END_C_DECLS }
+#else
+# define __BEGIN_C_DECLS /* empty */
+# define __END_C_DECLS /* empty */
+#endif
+/* End of C++ preparations */
+
+
+
+
+
+/* Actual header contants (the above were for the Pre-processor). */
+__BEGIN_C_DECLS /* From C++ preparations */
+
+
+
+
+
+/* Functions */
+int
+gal_array_name_recognized(char *name);
+
+gal_data_t *
+gal_array_read(char *filename, char *extension, size_t minmapsize);
+
+gal_data_t *
+gal_array_read_to_type(char *filename, char *extension, uint8_t type,
+ size_t minmapsize);
+
+gal_data_t *
+gal_array_read_one_ch(char *filename, char *extension, size_t minmapsize);
+
+gal_data_t *
+gal_array_read_one_ch_to_type(char *filename, char *extension, uint8_t type,
+ size_t minmapsize);
+
+
+__END_C_DECLS /* From C++ preparations */
+
+#endif /* __GAL_TIFF_H__ */
diff --git a/lib/gnuastro/binary.h b/lib/gnuastro/binary.h
index bac8b7c..4ae9357 100644
--- a/lib/gnuastro/binary.h
+++ b/lib/gnuastro/binary.h
@@ -104,4 +104,4 @@ gal_binary_fill_holes(gal_data_t *input, int connectivity,
size_t maxsize);
__END_C_DECLS /* From C++ preparations */
-#endif /* __GAL_DATA_H__ */
+#endif /* __GAL_BINARY_H__ */
diff --git a/lib/gnuastro/blank.h b/lib/gnuastro/blank.h
index 674496f..eab1fbf 100644
--- a/lib/gnuastro/blank.h
+++ b/lib/gnuastro/blank.h
@@ -96,6 +96,9 @@ char *
gal_blank_as_string(uint8_t type, int width);
int
+gal_blank_is(void *pointer, uint8_t type);
+
+int
gal_blank_present(gal_data_t *input, int updateflag);
gal_data_t *
diff --git a/lib/gnuastro/dimension.h b/lib/gnuastro/dimension.h
index 2cb434b..90c2c41 100644
--- a/lib/gnuastro/dimension.h
+++ b/lib/gnuastro/dimension.h
@@ -26,6 +26,7 @@ along with Gnuastro. If not, see
<http://www.gnu.org/licenses/>.
/* Include other headers if necessary here. Note that other header files
must be included before the C++ preparations below */
#include <gnuastro/data.h>
+#include <gnuastro/blank.h>
/* C++ Preparations */
#undef __BEGIN_C_DECLS
diff --git a/lib/gnuastro/eps.h b/lib/gnuastro/eps.h
new file mode 100644
index 0000000..ab8ed3d
--- /dev/null
+++ b/lib/gnuastro/eps.h
@@ -0,0 +1,75 @@
+/*********************************************************************
+eps -- functions to write EPS files.
+This is part of GNU Astronomy Utilities (Gnuastro) package.
+
+Original author:
+ Mohammad Akhlaghi <mohammad@akhlaghi.org>
+Contributing author(s):
+Copyright (C) 2015-2018, Free Software Foundation, Inc.
+
+Gnuastro is free software: you can redistribute it and/or modify it
+under the terms of the GNU General Public License as published by the
+Free Software Foundation, either version 3 of the License, or (at your
+option) any later version.
+
+Gnuastro is distributed in the hope that it will be useful, but
+WITHOUT ANY WARRANTY; without even the implied warranty of
+MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the GNU
+General Public License for more details.
+
+You should have received a copy of the GNU General Public License
+along with Gnuastro. If not, see <http://www.gnu.org/licenses/>.
+**********************************************************************/
+#ifndef __GAL_EPS_H__
+#define __GAL_EPS_H__
+
+
+/* Include other headers if necessary here. Note that other header files
+ must be included before the C++ preparations below */
+#include <gnuastro/data.h>
+
+
+
+/* C++ Preparations */
+#undef __BEGIN_C_DECLS
+#undef __END_C_DECLS
+#ifdef __cplusplus
+# define __BEGIN_C_DECLS extern "C" {
+# define __END_C_DECLS }
+#else
+# define __BEGIN_C_DECLS /* empty */
+# define __END_C_DECLS /* empty */
+#endif
+/* End of C++ preparations */
+
+
+
+
+
+/* Actual header contants (the above were for the Pre-processor). */
+__BEGIN_C_DECLS /* From C++ preparations */
+
+
+
+
+
+/* Functions */
+int
+gal_eps_name_is_eps(char *name);
+
+int
+gal_eps_suffix_is_eps(char *name);
+
+void
+gal_eps_to_pt(float widthincm, size_t *dsize, size_t *w_h_in_pt);
+
+void
+gal_eps_write(gal_data_t *in, char *filename, float widthincm,
+ uint32_t borderwidth, int hex, int forpdf);
+
+
+
+
+__END_C_DECLS /* From C++ preparations */
+
+#endif /* __GAL_TIFF_H__ */
diff --git a/lib/gnuastro/fits.h b/lib/gnuastro/fits.h
index 058ef12..370355f 100644
--- a/lib/gnuastro/fits.h
+++ b/lib/gnuastro/fits.h
@@ -210,13 +210,11 @@ gal_fits_img_info(fitsfile *fptr, int *type, size_t
*ndim, size_t **dsize,
char **name, char **unit);
gal_data_t *
-gal_fits_img_read(char *filename, char *hdu, size_t minmapsize,
- size_t hstartwcs, size_t hendwcs);
+gal_fits_img_read(char *filename, char *hdu, size_t minmapsize);
gal_data_t *
gal_fits_img_read_to_type(char *inputname, char *hdu, uint8_t type,
- size_t minmapsize, size_t hstartwcs,
- size_t hendwcs);
+ size_t minmapsize);
gal_data_t *
gal_fits_img_read_kernel(char *filename, char *hdu, size_t minmapsize);
diff --git a/lib/gnuastro/jpeg.h b/lib/gnuastro/jpeg.h
new file mode 100644
index 0000000..52c1cc7
--- /dev/null
+++ b/lib/gnuastro/jpeg.h
@@ -0,0 +1,75 @@
+/*********************************************************************
+jpeg -- functions to read and write JPEG files.
+This is part of GNU Astronomy Utilities (Gnuastro) package.
+
+Original author:
+ Mohammad Akhlaghi <mohammad@akhlaghi.org>
+Contributing author(s):
+Copyright (C) 2015-2018, Free Software Foundation, Inc.
+
+Gnuastro is free software: you can redistribute it and/or modify it
+under the terms of the GNU General Public License as published by the
+Free Software Foundation, either version 3 of the License, or (at your
+option) any later version.
+
+Gnuastro is distributed in the hope that it will be useful, but
+WITHOUT ANY WARRANTY; without even the implied warranty of
+MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the GNU
+General Public License for more details.
+
+You should have received a copy of the GNU General Public License
+along with Gnuastro. If not, see <http://www.gnu.org/licenses/>.
+**********************************************************************/
+#ifndef __GAL_JPEG_H__
+#define __GAL_JPEG_H__
+
+
+/* Include other headers if necessary here. Note that other header files
+ must be included before the C++ preparations below */
+#include <gnuastro/data.h>
+
+
+
+/* C++ Preparations */
+#undef __BEGIN_C_DECLS
+#undef __END_C_DECLS
+#ifdef __cplusplus
+# define __BEGIN_C_DECLS extern "C" {
+# define __END_C_DECLS }
+#else
+# define __BEGIN_C_DECLS /* empty */
+# define __END_C_DECLS /* empty */
+#endif
+/* End of C++ preparations */
+
+
+
+
+
+/* Actual header contants (the above were for the Pre-processor). */
+__BEGIN_C_DECLS /* From C++ preparations */
+
+
+
+
+
+/* Functions */
+int
+gal_jpeg_name_is_jpeg(char *name);
+
+int
+gal_jpeg_suffix_is_jpeg(char *name);
+
+gal_data_t *
+gal_jpeg_read(char *filename, size_t minmapsize);
+
+void
+gal_jpeg_write(gal_data_t *in, char *filename, uint8_t quality,
+ float widthincm);
+
+
+
+
+__END_C_DECLS /* From C++ preparations */
+
+#endif /* __GAL_TIFF_H__ */
diff --git a/lib/gnuastro/label.h b/lib/gnuastro/label.h
new file mode 100644
index 0000000..e73b7d8
--- /dev/null
+++ b/lib/gnuastro/label.h
@@ -0,0 +1,74 @@
+/*********************************************************************
+label -- Work on labeled (positive integer valued) datasets.
+This is part of GNU Astronomy Utilities (Gnuastro) package.
+
+Original author:
+ Mohammad Akhlaghi <mohammad@akhlaghi.org>
+Contributing author(s):
+Copyright (C) 2018, Free Software Foundation, Inc.
+
+Gnuastro is free software: you can redistribute it and/or modify it
+under the terms of the GNU General Public License as published by the
+Free Software Foundation, either version 3 of the License, or (at your
+option) any later version.
+
+Gnuastro is distributed in the hope that it will be useful, but
+WITHOUT ANY WARRANTY; without even the implied warranty of
+MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the GNU
+General Public License for more details.
+
+You should have received a copy of the GNU General Public License
+along with Gnuastro. If not, see <http://www.gnu.org/licenses/>.
+**********************************************************************/
+#ifndef __GAL_LABEL_H__
+#define __GAL_LABEL_H__
+
+/* Include other headers if necessary here. Note that other header files
+ must be included before the C++ preparations below */
+#include <gnuastro/data.h>
+
+/* C++ Preparations */
+#undef __BEGIN_C_DECLS
+#undef __END_C_DECLS
+#ifdef __cplusplus
+# define __BEGIN_C_DECLS extern "C" {
+# define __END_C_DECLS }
+#else
+# define __BEGIN_C_DECLS /* empty */
+# define __END_C_DECLS /* empty */
+#endif
+/* End of C++ preparations */
+
+
+
+/* Actual header contants (the above were for the Pre-processor). */
+__BEGIN_C_DECLS /* From C++ preparations */
+
+
+
+
+
+/* Constants for the clump over-segmentation. */
+#define GAL_LABEL_INIT -1
+#define GAL_LABEL_RIVER -2
+#define GAL_LABEL_TMPCHECK -3
+
+
+
+
+
+/* Functions. */
+size_t
+gal_label_oversegment(gal_data_t *input, gal_data_t *indexs,
+ gal_data_t *label, size_t *topinds);
+
+void
+gal_label_grow_indexs(gal_data_t *labels, gal_data_t *indexs, int withrivers,
+ int connectivity);
+
+
+
+
+__END_C_DECLS /* From C++ preparations */
+
+#endif /* __GAL_LABEL_H__ */
diff --git a/bin/mkcatalog/upperlimit.h b/lib/gnuastro/pdf.h
similarity index 50%
copy from bin/mkcatalog/upperlimit.h
copy to lib/gnuastro/pdf.h
index 143252f..033a8e4 100644
--- a/bin/mkcatalog/upperlimit.h
+++ b/lib/gnuastro/pdf.h
@@ -1,6 +1,6 @@
/*********************************************************************
-MakeCatalog - Make a catalog from an input and labeled image.
-MakeCatalog is part of GNU Astronomy Utilities (Gnuastro) package.
+pdf -- functions to write PDF files.
+This is part of GNU Astronomy Utilities (Gnuastro) package.
Original author:
Mohammad Akhlaghi <mohammad@akhlaghi.org>
@@ -20,10 +20,53 @@ General Public License for more details.
You should have received a copy of the GNU General Public License
along with Gnuastro. If not, see <http://www.gnu.org/licenses/>.
**********************************************************************/
-#ifndef UPPERLIMIT_H
-#define UPPERLIMIT_H
+#ifndef __GAL_PDF_H__
+#define __GAL_PDF_H__
+
+
+/* Include other headers if necessary here. Note that other header files
+ must be included before the C++ preparations below */
+#include <gnuastro/data.h>
-void
-upperlimit_calculate(struct mkcatalog_passparams *pp);
+
+/* C++ Preparations */
+#undef __BEGIN_C_DECLS
+#undef __END_C_DECLS
+#ifdef __cplusplus
+# define __BEGIN_C_DECLS extern "C" {
+# define __END_C_DECLS }
+#else
+# define __BEGIN_C_DECLS /* empty */
+# define __END_C_DECLS /* empty */
#endif
+/* End of C++ preparations */
+
+
+
+
+
+/* Actual header contants (the above were for the Pre-processor). */
+__BEGIN_C_DECLS /* From C++ preparations */
+
+
+
+
+
+/* Functions */
+int
+gal_pdf_name_is_pdf(char *name);
+
+int
+gal_pdf_suffix_is_pdf(char *name);
+
+void
+gal_pdf_write(gal_data_t *in, char *filename, float widthincm,
+ uint32_t borderwidth);
+
+
+
+
+__END_C_DECLS /* From C++ preparations */
+
+#endif /* __GAL_TIFF_H__ */
diff --git a/lib/gnuastro/statistics.h b/lib/gnuastro/statistics.h
index 510d905..55a732d 100644
--- a/lib/gnuastro/statistics.h
+++ b/lib/gnuastro/statistics.h
@@ -62,6 +62,7 @@ enum is_sorted_outputs
{
GAL_STATISTICS_SORTED_NOT, /* ==0 by C standard. */
+ GAL_STATISTICS_SORTED_INVALID,
GAL_STATISTICS_SORTED_INCREASING,
GAL_STATISTICS_SORTED_DECREASING,
};
diff --git a/lib/gnuastro/tiff.h b/lib/gnuastro/tiff.h
new file mode 100644
index 0000000..2e2b008
--- /dev/null
+++ b/lib/gnuastro/tiff.h
@@ -0,0 +1,74 @@
+/*********************************************************************
+tiff -- functions to read and write TIFF files.
+This is part of GNU Astronomy Utilities (Gnuastro) package.
+
+Original author:
+ Mohammad Akhlaghi <mohammad@akhlaghi.org>
+Contributing author(s):
+Copyright (C) 2018, Free Software Foundation, Inc.
+
+Gnuastro is free software: you can redistribute it and/or modify it
+under the terms of the GNU General Public License as published by the
+Free Software Foundation, either version 3 of the License, or (at your
+option) any later version.
+
+Gnuastro is distributed in the hope that it will be useful, but
+WITHOUT ANY WARRANTY; without even the implied warranty of
+MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the GNU
+General Public License for more details.
+
+You should have received a copy of the GNU General Public License
+along with Gnuastro. If not, see <http://www.gnu.org/licenses/>.
+**********************************************************************/
+#ifndef __GAL_TIFF_H__
+#define __GAL_TIFF_H__
+
+
+/* Include other headers if necessary here. Note that other header files
+ must be included before the C++ preparations below */
+#include <gnuastro/list.h>
+
+
+
+/* C++ Preparations */
+#undef __BEGIN_C_DECLS
+#undef __END_C_DECLS
+#ifdef __cplusplus
+# define __BEGIN_C_DECLS extern "C" {
+# define __END_C_DECLS }
+#else
+# define __BEGIN_C_DECLS /* empty */
+# define __END_C_DECLS /* empty */
+#endif
+/* End of C++ preparations */
+
+
+
+
+
+/* Actual header contants (the above were for the Pre-processor). */
+__BEGIN_C_DECLS /* From C++ preparations */
+
+
+
+
+
+/* Functions */
+int
+gal_tiff_name_is_tiff(char *name);
+
+int
+gal_tiff_suffix_is_tiff(char *name);
+
+size_t
+gal_tiff_dir_string_read(char *string);
+
+gal_data_t *
+gal_tiff_read(char *filename, size_t dir, size_t minmapsize);
+
+
+
+
+__END_C_DECLS /* From C++ preparations */
+
+#endif /* __GAL_TIFF_H__ */
diff --git a/bin/convertt/jpeg.c b/lib/jpeg.c
similarity index 78%
rename from bin/convertt/jpeg.c
rename to lib/jpeg.c
index dd8989b..f89c6d0 100644
--- a/bin/convertt/jpeg.c
+++ b/lib/jpeg.c
@@ -1,6 +1,6 @@
/*********************************************************************
-ConvertType - Convert between various types of files.
-ConvertType is part of GNU Astronomy Utilities (Gnuastro) package.
+jpeg -- functions to read and write JPEG files.
+This is part of GNU Astronomy Utilities (Gnuastro) package.
Original author:
Mohammad Akhlaghi <mohammad@akhlaghi.org>
@@ -33,10 +33,7 @@ along with Gnuastro. If not, see
<http://www.gnu.org/licenses/>.
#endif
#include <gnuastro/list.h>
-#include <gnuastro/fits.h>
-
-#include "main.h"
-#include "jpeg.h"
+#include <gnuastro/jpeg.h>
@@ -48,7 +45,7 @@ along with Gnuastro. If not, see
<http://www.gnu.org/licenses/>.
************** Acceptable JPEG names **************
*************************************************************/
int
-nameisjpeg(char *name)
+gal_jpeg_name_is_jpeg(char *name)
{
size_t len;
len=strlen(name);
@@ -70,7 +67,7 @@ nameisjpeg(char *name)
int
-nameisjpegsuffix(char *name)
+gal_jpeg_suffix_is_jpeg(char *name)
{
if (strcmp(name, "jpg") == 0 || strcmp(name, ".jpg") == 0
|| strcmp(name, "JPG") == 0 || strcmp(name, ".JPG") == 0
@@ -102,10 +99,12 @@ nameisjpegsuffix(char *name)
-#ifdef HAVE_LIBJPEG
+
+
/*************************************************************
************** Read a JPEG image **************
*************************************************************/
+#ifdef HAVE_LIBJPEG
/* Read the example.c in libjpeg's source code to understand the
details of what is going on here. */
struct my_error_mgr
@@ -135,7 +134,7 @@ jpeg_error_exit(j_common_ptr cinfo)
-void
+static void
makejsample(JSAMPLE **a, size_t size)
{
JSAMPLE *jsarr;
@@ -159,7 +158,7 @@ makejsample(JSAMPLE **a, size_t size)
-unsigned char **
+static unsigned char **
readjpg(char *inname, size_t *outs0, size_t *outs1, size_t *numcolors)
{
FILE * infile;
@@ -238,6 +237,7 @@ readjpg(char *inname, size_t *outs0, size_t *outs1, size_t
*numcolors)
return all;
}
+#endif /* HAVE_LIBJPEG */
@@ -245,10 +245,12 @@ readjpg(char *inname, size_t *outs0, size_t *outs1,
size_t *numcolors)
/* Read each color channel of a JPEG image as a separate array and put them
in a linked list of data-structures. */
-size_t
-jpeg_read_to_ll(char *filename, gal_data_t **list, size_t minmapsize)
+gal_data_t *
+gal_jpeg_read(char *filename, size_t minmapsize)
{
+#ifdef HAVE_LIBJPEG
char *name;
+ gal_data_t *out=NULL;
size_t ndim=2, dsize[2];
unsigned char **allcolors;
size_t i, s0, s1, numcolors;
@@ -263,7 +265,7 @@ jpeg_read_to_ll(char *filename, gal_data_t **list, size_t
minmapsize)
dsize[1]=s1;
if( asprintf(&name, "JPEG_CH_%zu", i+1)<0 )
error(EXIT_FAILURE, 0, "%s: asprintf allocation", __func__);
- gal_list_data_add_alloc(list, allcolors[i], GAL_TYPE_UINT8, ndim,
+ gal_list_data_add_alloc(&out, allcolors[i], GAL_TYPE_UINT8, ndim,
dsize, NULL, 0, minmapsize, name, NULL, NULL);
free(name);
}
@@ -275,7 +277,14 @@ jpeg_read_to_ll(char *filename, gal_data_t **list, size_t
minmapsize)
free(allcolors);
/* Return the number of color channels. */
- return numcolors;
+ return out;
+#else
+ error(EXIT_FAILURE, 0, "%s: libjpeg was not found during the "
+ "configuration of %s on this system. To read from JPEG files, "
+ "libjpeg is required. Please install libjpeg and configure, make "
+ "and install %s again", __func__, PACKAGE_STRING, PACKAGE_STRING);
+ return NULL;
+#endif
}
@@ -300,28 +309,36 @@ jpeg_read_to_ll(char *filename, gal_data_t **list, size_t
minmapsize)
/*************************************************************
************** Write a JPEG image **************
*************************************************************/
+#ifdef HAVE_LIBJPEG
static void
-jpeg_write_array(JSAMPLE *jsr, struct converttparams *p)
+jpeg_write_array(JSAMPLE *jsr, gal_data_t *in, char *filename,
+ uint8_t quality, float widthincm)
{
JSAMPROW r[1];
FILE * outfile;
int row_stride=0, c;
+ size_t *dsize=in->dsize;
struct jpeg_error_mgr jerr;
- size_t *dsize=p->chll->dsize;
struct jpeg_compress_struct cinfo;
+ size_t numch=gal_list_data_number(in);
+
+ /* A small sanity check. */
+ if(quality > 100)
+ error(EXIT_FAILURE, 0, "%s: quality value %u not acceptable. It must be "
+ "a value between zero and 100 (inclusive)", __func__, quality);
/* Begin the JPEG writing, following libjpeg's example.c */
cinfo.err = jpeg_std_error(&jerr);
jpeg_create_compress(&cinfo);
errno=0;
- if ((outfile = fopen(p->cp.output, "wb")) == NULL)
- error(EXIT_FAILURE, errno, "%s", p->cp.output);
+ if ((outfile = fopen(filename, "wb")) == NULL)
+ error(EXIT_FAILURE, errno, "%s", filename);
jpeg_stdio_dest(&cinfo, outfile);
cinfo.image_width = dsize[1];
cinfo.image_height = dsize[0];
- switch(p->numch)
+ switch(numch)
{
case 1:
row_stride=dsize[1];
@@ -341,13 +358,13 @@ jpeg_write_array(JSAMPLE *jsr, struct converttparams *p)
default:
error(EXIT_FAILURE, 0, "%s: a bug! The number of channels is not 1, 3 "
"or 4, but %zu. This should not happen. Please contact us so we "
- "can fix the problem", __func__, p->numch);
+ "can fix the problem", __func__, numch);
}
jpeg_set_defaults(&cinfo);
- jpeg_set_quality(&cinfo, p->quality, TRUE);
+ jpeg_set_quality(&cinfo, quality, TRUE);
cinfo.density_unit=1;
- cinfo.Y_density=cinfo.X_density=dsize[1]/(p->widthincm/2.54);
+ cinfo.Y_density=cinfo.X_density=dsize[1]/(widthincm/2.54);
jpeg_start_compress(&cinfo, TRUE);
/* cinfo.next_scanline is 'unsigned int' */
@@ -362,41 +379,48 @@ jpeg_write_array(JSAMPLE *jsr, struct converttparams *p)
fclose(outfile);
jpeg_destroy_compress(&cinfo);
}
+#endif /* HAVE_LIBJPEG */
void
-jpeg_write(struct converttparams *p)
+gal_jpeg_write(gal_data_t *in, char *filename, uint8_t quality,
+ float widthincm)
{
+#ifdef HAVE_LIBJPEG
JSAMPLE *jsr;
gal_data_t *channel;
unsigned char *colors[4];
- size_t i, pixel, color, numch=p->numch;
+ size_t i, pixel, color;
+ size_t numch=gal_list_data_number(in);
- /* A small sanity check */
- if(p->numch==2 || p->numch>4)
+ /* Small sanity checks. */
+ if(numch==2 || numch>4)
error(EXIT_FAILURE, 0, "%s: only 1, 3, and 4 color channels are "
- "acceptable", __func__);
+ "acceptable, input is a list of %zu data sets", __func__, numch);
+ if(in->type!=GAL_TYPE_UINT8)
+ error(EXIT_FAILURE, 0, "%s: input has a `%s' type, but JPEG images can "
+ "only have a `uint8' type", __func__, gal_type_name(in->type, 1));
/* Make sure the JSAMPLE is 8bits, then allocate the necessary space
based on the number of channels. */
if(sizeof *jsr!=1)
error(EXIT_FAILURE, 0, "%s: JSAMPLE has to be 8bit", __func__);
errno=0;
- jsr=malloc(numch * p->chll->size * sizeof *jsr);
+ jsr=malloc(numch * in->size * sizeof *jsr);
if(jsr==NULL)
error(EXIT_FAILURE, errno, "%s: allocating %zu bytes for jsr",
- __func__, numch * p->chll->size * sizeof *jsr );
+ __func__, numch * in->size * sizeof *jsr );
/* Set the pointers to each color. */
i=0;
- for(channel=p->chll; channel!=NULL; channel=channel->next)
+ for(channel=in; channel!=NULL; channel=channel->next)
colors[i++]=channel->array;
/* Write the different colors into jsr. */
- for(pixel=0; pixel<p->chll->size; ++pixel)
+ for(pixel=0; pixel<in->size; ++pixel)
for(color=0;color<numch;++color)
{
jsr[pixel*numch+color] = colors[color][pixel];
@@ -407,7 +431,12 @@ jpeg_write(struct converttparams *p)
}
/* Write jsr to a JPEG image and clean up. */
- jpeg_write_array(jsr, p);
+ jpeg_write_array(jsr, in, filename, quality, widthincm);
free(jsr);
-}
+#else
+ error(EXIT_FAILURE, 0, "%s: libjpeg was not found during the "
+ "configuration of %s on this system. To write JPEG files, libjpeg "
+ "is required. Please install libjpeg, then configure, make and "
+ "install %s again", __func__, PACKAGE_STRING, PACKAGE_STRING);
#endif /* HAVE_LIBJPEG */
+}
diff --git a/lib/label.c b/lib/label.c
new file mode 100644
index 0000000..538f704
--- /dev/null
+++ b/lib/label.c
@@ -0,0 +1,529 @@
+/*********************************************************************
+label -- Work on labeled (integer valued) datasets.
+This is part of GNU Astronomy Utilities (Gnuastro) package.
+
+Original author:
+ Mohammad Akhlaghi <mohammad@akhlaghi.org>
+Contributing author(s):
+Copyright (C) 2018, Free Software Foundation, Inc.
+
+Gnuastro is free software: you can redistribute it and/or modify it
+under the terms of the GNU General Public License as published by the
+Free Software Foundation, either version 3 of the License, or (at your
+option) any later version.
+
+Gnuastro is distributed in the hope that it will be useful, but
+WITHOUT ANY WARRANTY; without even the implied warranty of
+MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the GNU
+General Public License for more details.
+
+You should have received a copy of the GNU General Public License
+along with Gnuastro. If not, see <http://www.gnu.org/licenses/>.
+**********************************************************************/
+#include <config.h>
+
+#include <stdio.h>
+#include <errno.h>
+#include <error.h>
+#include <stdlib.h>
+
+#include <gnuastro/list.h>
+#include <gnuastro/qsort.h>
+#include <gnuastro/label.h>
+#include <gnuastro/dimension.h>
+
+
+
+
+
+
+/****************************************************************
+ ***************** Over segmentation ********************
+ ****************************************************************/
+/* Over-segment the region specified by its indexs into peaks and their
+ respective regions (clumps). This is very similar to the immersion
+ method of Vincent & Soille(1991), but here, we will not separate the
+ image into layers, instead, we will work based on the ordered flux
+ values. If a certain pixel (at a certain level) has no neighbors, it is
+ a local maximum and will be assigned a new label. If it has a labeled
+ neighbor, it will take that label and if there is more than one
+ neighboring labeled region that pixel will be a `river` pixel.
+
+ DON'T FORGET: SET THE FLAGS FOR CONV EQUAL TO INPUT IN SEGMENT.
+
+*/
+size_t
+gal_label_oversegment(gal_data_t *input, gal_data_t *indexs,
+ gal_data_t *label, size_t *topinds)
+{
+ size_t ndim=input->ndim;
+
+ float *arr=input->array;
+ gal_list_sizet_t *Q=NULL, *cleanup=NULL;
+ size_t *a, *af, ind, *dsize=input->dsize;
+ size_t *dinc=gal_dimension_increment(ndim, dsize);
+ int32_t n1, nlab, rlab, curlab=1, *clabel=label->array;
+
+ /*********************************************
+ For checks and debugging:*
+ gal_data_t *crop;
+ size_t extcount=1;
+ int32_t *cr, *crf;
+ size_t checkdsize[2]={10,10};
+ size_t checkstart[2]={50,145};
+ char *filename="clumpbuild.fits";
+ size_t checkstartind=gal_dimension_coord_to_index(2, dsize, checkstart);
+ gal_data_t *tile=gal_data_alloc(gal_data_ptr_increment(arr, checkstartind,
+ input->type),
+ GAL_TYPE_INVALID, 2, checkdsize,
+ NULL, 0, 0, NULL, NULL, NULL);
+ tile->block=input;
+ gal_checkset_writable_remove(filename, 0, 0);
+ if(p->cp.numthreads!=1)
+ error(EXIT_FAILURE, 0, "in the debugging mode of `clumps_oversegment' "
+ "only one thread must be used");
+ crop=gal_data_copy(tile);
+ gal_fits_img_write(crop, filename, NULL, PROGRAM_NAME);
+ gal_data_free(crop);
+ printf("blank: %u\nriver: %u\ntmpcheck: %u\ninit: %u\n",
+ (int32_t)GAL_BLANK_INT32, (int32_t)GAL_LABEL_RIVER,
+ (int32_t)GAL_LABEL_TMPCHECK, (int32_t)GAL_LABEL_INIT);
+ tile->array=gal_tile_block_relative_to_other(tile, clabel);
+ tile->block=clabel;
+ **********************************************/
+
+
+ /* If the size of the indexs is zero, then this function is pointless. */
+ if(indexs->size==0) return 0;
+
+
+ /* Sort the given indexs based on their flux (`gal_qsort_index_arr' is
+ defined as static in `gnuastro/qsort.h') */
+ gal_qsort_index_arr=input->array;
+ qsort(indexs->array, indexs->size, sizeof(size_t),
+ gal_qsort_index_float_decreasing);
+
+
+ /* Initialize the region we want to over-segment. */
+ af=(a=indexs->array)+indexs->size;
+ do clabel[*a]=GAL_LABEL_INIT; while(++a<af);
+
+
+ /* Go over all the given indexs and pull out the clumps. */
+ af=(a=indexs->array)+indexs->size;
+ do
+ /* When regions of a constant flux or masked regions exist, some later
+ indexs (although they have same flux) will be filled before hand. If
+ they are done, there is no need to do them again. */
+ if(clabel[*a]==GAL_LABEL_INIT)
+ {
+ /* It might happen where one or multiple regions of the pixels
+ under study have the same flux. So two equal valued pixels of
+ two separate (but equal flux) regions will fall immediately
+ after each other in the sorted list of indexs and we have to
+ account for this.
+
+ Therefore, if we see that the next pixel in the index list has
+ the same flux as this one, it does not guarantee that it should
+ be given the same label. Similar to the breadth first search
+ algorithm for finding connected components, we will search all
+ the neighbours and the neighbours of those neighbours that have
+ the same flux of this pixel to see if they touch any label or
+ not and to finally give them all the same label. */
+ if( (a+1)<af && arr[*a]==arr[*(a+1)] )
+ {
+ /* Label of first neighbor found. */
+ n1=0;
+
+ /* A small sanity check. */
+ if(Q!=NULL || cleanup!=NULL)
+ error(EXIT_FAILURE, 0, "%s: a bug! Please contact us at %s so "
+ "we can fix this problem. `Q' and `cleanup' should be "
+ "NULL but while checking the equal flux regions they "
+ "aren't", __func__, PACKAGE_BUGREPORT);
+
+ /* Add this pixel to a queue. */
+ gal_list_sizet_add(&Q, *a);
+ gal_list_sizet_add(&cleanup, *a);
+ clabel[*a] = GAL_LABEL_TMPCHECK;
+
+ /* Find all the pixels that have the same flux and are
+ connected. */
+ while(Q!=NULL)
+ {
+ /* Pop an element from the queue. */
+ ind=gal_list_sizet_pop(&Q);
+
+ /* Look at the neighbors and see if we already have a
+ label. */
+ GAL_DIMENSION_NEIGHBOR_OP(ind, ndim, dsize, ndim, dinc,
+ {
+ /* If it is already decided to be a river, then stop
+ looking at the neighbors. */
+ if(n1!=GAL_LABEL_RIVER)
+ {
+ /* For easy reading. */
+ nlab=clabel[ nind ];
+
+ /* This neighbor's label isn't zero. */
+ if(nlab)
+ {
+ /* If this neighbor has not been labeled yet
+ and has an equal flux, add it to the queue
+ to expand the studied region.*/
+ if( nlab==GAL_LABEL_INIT && arr[nind]==arr[*a] )
+ {
+ clabel[nind]=GAL_LABEL_TMPCHECK;
+ gal_list_sizet_add(&Q, nind);
+ gal_list_sizet_add(&cleanup, nind);
+ }
+ else
+ n1=( nlab>0
+
+ /* If this neighbor has a positive
+ nlab, it belongs to another object,
+ so if `n1' has not been set for the
+ whole region (n1==0), put `nlab'
+ into `n1'. If `n1' has been set and
+ is different from `nlab' then this
+ whole equal flux region should be a
+ wide river because it is connecting
+ two connected regions.*/
+ ? ( n1
+ ? (n1==nlab ? n1 : GAL_LABEL_RIVER)
+ : nlab )
+
+ /* If the data has blank pixels (recall
+ that blank in int32 is negative),
+ see if the neighbor is blank and if
+ so, set the label to a river. Since
+ the flag checking can be done
+ outside this loop, for datasets with
+ no blank element this last step will
+ be completley ignored. */
+ : ( ( (input->flag
+ & GAL_DATA_FLAG_HASBLANK)
+ && nlab==GAL_BLANK_INT32 )
+ ? GAL_LABEL_RIVER : n1 ) );
+ }
+
+ /* If this neigbour has a label of zero, then we
+ are on the edge of the indexed region (the
+ neighbor is not in the initial list of pixels
+ to segment). When over-segmenting the noise and
+ the detections, `clabel' is zero for the parts
+ of the image that we are not interested in
+ here. */
+ else clabel[*a]=GAL_LABEL_RIVER;
+ }
+ } );
+ }
+
+ /* Set the label that is to be given to this equal flux
+ region. If `n1' was set to any value, then that label should
+ be used for the whole region. Otherwise, this is a new
+ label, see the case for a non-flat region. */
+ if(n1) rlab = n1;
+ else
+ {
+ rlab = curlab++;
+ if( topinds ) /* This is a local maximum of */
+ topinds[rlab]=*a; /* this region, save its index. */
+ }
+
+ /* Give the same label to the whole connected equal flux
+ region, except those that might have been on the side of
+ the image and were a river pixel. */
+ while(cleanup!=NULL)
+ {
+ ind=gal_list_sizet_pop(&cleanup);
+ /* If it was on the sides of the image, it has been
+ changed to a river pixel. */
+ if( clabel[ ind ]==GAL_LABEL_TMPCHECK ) clabel[ ind ]=rlab;
+ }
+ }
+
+ /* The flux of this pixel is not the same as the next sorted
+ flux, so simply find the label for this object. */
+ else
+ {
+ /* `n1' is the label of the first labeled neighbor found, so
+ we'll initialize it to zero. */
+ n1=0;
+
+ /* Go over all the fully connected neighbors of this pixel and
+ see if all the neighbors (with maximum connectivity: the
+ number of dimensions) that have a non-macro value belong to
+ one label or not. If the pixel is neighboured by more than
+ one label, set it as a river pixel. Also if it is touching a
+ zero valued pixel (which does not belong to this object),
+ set it as a river pixel.*/
+ GAL_DIMENSION_NEIGHBOR_OP(*a, ndim, dsize, ndim, dinc,
+ {
+ /* When `n1' has already been set as a river, there is no
+ point in looking at the other neighbors. */
+ if(n1!=GAL_LABEL_RIVER)
+ {
+ /* For easy reading. */
+ nlab=clabel[ nind ];
+
+ /* If this neighbor is on a non-processing label, then
+ set the first neighbor accordingly. Note that we
+ also want the zero valued neighbors (detections if
+ working on sky, and sky if working on detection):
+ we want rivers between the two domains. */
+ n1 = ( nlab
+
+ /* nlab is non-zero. */
+ ? ( nlab>0
+
+ /* Neighbor has a meaningful label, so
+ check with any previously found labeled
+ neighbors. */
+ ? ( n1
+ ? ( nlab==n1 ? n1 : GAL_LABEL_RIVER )
+ : nlab )
+
+ /* If the dataset has blank values and this
+ neighbor is blank, then the pixel should
+ be a river. Note that the blank checking
+ can be optimized out, so if the input
+ doesn't have blank values,
+ `nlab==GAL_BLANK_INT32' will never be
+ checked. */
+ : ( (input->flag & GAL_DATA_FLAG_HASBLANK)
+ && nlab==GAL_BLANK_INT32
+ ? GAL_LABEL_RIVER : n1 ) )
+
+ /* `nlab==0' (the neighbor lies in the other
+ domain (sky or detections). To avoid the
+ different domains touching, this pixel
+ should be a river. */
+ : GAL_LABEL_RIVER );
+ }
+ });
+
+ /* Either assign a new label to this pixel, or give it the one
+ of its neighbors. If n1 equals zero, then this is a new
+ peak, and a new label should be created. But if n1!=0, it
+ is either a river pixel (has more than one labeled neighbor
+ and has been set to `GAL_LABEL_RIVER' before) or all its
+ neighbors have the same label. In both such cases, rlab
+ should be set to n1.*/
+ if(n1) rlab = n1;
+ else
+ {
+ rlab = curlab++;
+ if( topinds )
+ topinds[ rlab ]=*a;
+ }
+
+ /* Put the found label in the pixel. */
+ clabel[ *a ] = rlab;
+ }
+
+ /*********************************************
+ For checks and debugging:
+ if( *a / dsize[1] >= checkstart[0]
+ && *a / dsize[1] < checkstart[0] + checkdsize[0]
+ && *a % dsize[1] >= checkstart[1]
+ && *a % dsize[1] < checkstart[1] + checkdsize[1] )
+ {
+ printf("%zu (%zu: %zu, %zu): %u\n", ++extcount, *a,
+ (*a%dsize[1])-checkstart[1], (*a/dsize[1])-checkstart[0],
+ clabel[*a]);
+ crop=gal_data_copy(tile);
+ crf=(cr=crop->array)+crop->size;
+ do if(*cr==GAL_LABEL_RIVER) *cr=0; while(++cr<crf);
+ gal_fits_img_write(crop, filename, NULL, PROGRAM_NAME);
+ gal_data_free(crop);
+ }
+ **********************************************/
+ }
+ while(++a<af);
+
+ /*********************************************
+ For checks and debugging:
+ tile->array=NULL;
+ gal_data_free(tile);
+ printf("Total number of clumps: %u\n", curlab-1);
+ **********************************************/
+
+ /* Clean up. */
+ free(dinc);
+
+ /* Return the total number of clumps. */
+ return curlab-1;
+}
+
+
+
+
+
+/* Grow the given labels over the list of indexs. In over-segmentation
+ (watershed algorithm), the indexs have to be sorted by pixel value and
+ local maximums (that aren't touching any already labeled pixel) get a
+ separate label. However, here the final number of labels will not
+ change. All pixels that aren't directly touching a labeled pixel just
+ get pushed back to the start of the loop, and the loop iterates until
+ its size doesn't change any more. This is because in a generic scenario
+ some of the indexed pixels might not be reachable. As a result, it is
+ not necessary for the given indexs to be sorted here.
+
+ This function looks for positive-valued neighbors and will label a pixel
+ if it touches one. Therefore, it is is very important that only
+ pixels/labels that are intended for growth have positive values before
+ calling this function. Any non-positive (zero or negative) value will be
+ ignored by this function. Thus, it is recommended that while filling in
+ the `indexs' array values, you initialize all those pixels with
+ `GAL_LABEL_INIT', and set non-labeled pixels that you don't want to grow
+ to 0.
+
+ This function will write into both the input datasets. After this
+ function, some of the non-positive `labels' pixels will have a new
+ positive label and the number of useful elements in `indexs' will have
+ decreased. Those indexs that couldn't be labeled will remain inside
+ `indexs'. If `withrivers' is non-zero, then pixels that are immediately
+ touching more than one positive value will be given a `GAL_LABEL_RIVER'
+ label.
+
+ Note that the `indexs->array' is not re-allocated to its new size at the
+ end. But since `indexs->dsize[0]' and `indexs->size' have new values,
+ the extra elements just won't be used until they are ultimately freed by
+ `gal_data_free'.
+
+ Input:
+
+ labels: The labels array that must be operated on. The pixels that
+ must be "grown" must have the value `GAL_LABEL_INIT' (negative).
+
+ sorted_indexs: The indexs of the pixels that must be grown.
+
+ withrivers: as described above.
+
+ connectivity: connectivity to define neighbors for growth. */
+void
+gal_label_grow_indexs(gal_data_t *labels, gal_data_t *indexs, int withrivers,
+ int connectivity)
+{
+ int searchngb;
+ size_t *iarray=indexs->array;
+ int32_t n1, nlab, *olabel=labels->array;
+ size_t *s, *sf, thisround, ninds=indexs->size;
+ size_t *dinc=gal_dimension_increment(labels->ndim, labels->dsize);
+
+ /* Some basic sanity checks: */
+ if(labels->type!=GAL_TYPE_INT32)
+ error(EXIT_FAILURE, 0, "%s: `labels' has to have type of int32_t",
+ __func__);
+ if(indexs->type!=GAL_TYPE_SIZE_T)
+ error(EXIT_FAILURE, 0, "%s: `indexs' must be `size_t' type but it is "
+ "`%s'", __func__, gal_type_name(indexs->type,1));
+ if(indexs->ndim!=1)
+ error(EXIT_FAILURE, 0, "%s: `indexs' has to be a 1D array, but it is "
+ "%zuD", __func__, indexs->ndim);
+
+ /* The basic idea is this: after growing, not all the blank pixels are
+ necessarily filled, for example the pixels might belong to two regions
+ above the growth threshold. So the pixels in between them (which are
+ below the threshold will not ever be able to get a label). Therefore,
+ the safest way we can terminate the loop of growing the objects is to
+ stop it when the number of pixels left to fill in this round
+ (thisround) equals the number of blanks.
+
+ To start the loop, we set `thisround' to one more than the number of
+ indexed pixels. Note that it will be corrected immediately after the
+ loop has started, it is just important to pass the `while'. */
+ thisround=ninds+1;
+ while( thisround > ninds )
+ {
+ /* `thisround' will keep the number of pixels to be inspected in this
+ round. `ninds' will count the number of pixels left without a
+ label by the end of this round. Since `ninds' comes from the
+ previous loop (or outside, for the first round) it has to be saved
+ in `thisround' to begin counting a fresh. */
+ thisround=ninds;
+ ninds=0;
+
+ /* Go over all the available indexs. NOTE: while the `indexs->array'
+ pointer remains unchanged, `indexs->size' can/will change (get
+ smaller) in every loop. */
+ sf = (s=indexs->array) + indexs->size;
+ do
+ {
+ /* We'll begin by assuming the nearest neighbor of this pixel
+ has no label (has a value of 0). */
+ n1=0;
+
+ /* Check the neighbors of this pixel. Note that since this
+ macro has multiple loops within it, we can't use
+ break. We'll use the `searchngb' variable instead. */
+ searchngb=1;
+ GAL_DIMENSION_NEIGHBOR_OP(*s, labels->ndim, labels->dsize,
+ connectivity, dinc,
+ {
+ if(searchngb)
+ {
+ /* For easy reading. */
+ nlab = olabel[nind];
+
+ /* This neighbor's label is meaningful. */
+ if(nlab>0) /* This is a real label. */
+ {
+ if(n1) /* A prev. ngb label has been found. */
+ {
+ if( n1 != nlab ) /* Different label from */
+ { /* prevously found ngb for this pixel. */
+ n1=GAL_LABEL_RIVER;
+ searchngb=0;
+ }
+ }
+ else
+ { /* This is the first labeld neighbor found. */
+ n1=nlab;
+
+ /* If we want to completely fill in the region
+ (`withrivers==0'), then there is no point in
+ looking in other neighbors, the first
+ neighbor we find, is the one we'll use. */
+ if(!withrivers) searchngb=0;
+ }
+ }
+ }
+ } );
+
+ /* The loop over neighbors (above) finishes with three
+ possibilities:
+
+ n1==0 --> No labeled neighbor was found.
+ n1==GAL_LABEL_RIVER --> Connecting two labeled regions.
+ n1>0 --> Only has one neighbouring label.
+
+ The first one means that no neighbors were found and this
+ pixel should be kept for the next loop (we'll be growing the
+ objects pixel-layer by pixel-layer). In the other two cases,
+ we just need to write in the value of `n1'. */
+ if(n1)
+ {
+ /* Set the label. */
+ olabel[*s]=n1;
+
+ /* If this pixel is a river (can only happen when
+ `withrivers' is zero), keep it in the loop, because we
+ want the `indexs' dataset to contain all non-positive
+ (non-labeled) pixels, including rivers. */
+ if(n1==GAL_LABEL_RIVER)
+ iarray[ ninds++ ] = *s;
+ }
+ else
+ iarray[ ninds++ ] = *s;
+
+ /* Correct the size of the `indexs' dataset. */
+ indexs->size = indexs->dsize[0] = ninds;
+ }
+ while(++s<sf);
+ }
+
+ /* Clean up. */
+ free(dinc);
+}
diff --git a/lib/list.c b/lib/list.c
index b84e1c9..9af66fd 100644
--- a/lib/list.c
+++ b/lib/list.c
@@ -1199,7 +1199,7 @@ gal_list_data_add(gal_data_t **list, gal_data_t *newnode)
/* Set the next element of toadd and update what list points to.*/
toadd->next=*list;
- *list=toadd;
+ *list=newnode;
}
diff --git a/lib/pdf.c b/lib/pdf.c
new file mode 100644
index 0000000..794a9f5
--- /dev/null
+++ b/lib/pdf.c
@@ -0,0 +1,125 @@
+/*********************************************************************
+pdf -- functions to write PDF files.
+This is part of GNU Astronomy Utilities (Gnuastro) package.
+
+Original author:
+ Mohammad Akhlaghi <mohammad@akhlaghi.org>
+Contributing author(s):
+Copyright (C) 2015-2018, Free Software Foundation, Inc.
+
+Gnuastro is free software: you can redistribute it and/or modify it
+under the terms of the GNU General Public License as published by the
+Free Software Foundation, either version 3 of the License, or (at your
+option) any later version.
+
+Gnuastro is distributed in the hope that it will be useful, but
+WITHOUT ANY WARRANTY; without even the implied warranty of
+MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the GNU
+General Public License for more details.
+
+You should have received a copy of the GNU General Public License
+along with Gnuastro. If not, see <http://www.gnu.org/licenses/>.
+**********************************************************************/
+#include <config.h>
+
+#include <stdio.h>
+#include <errno.h>
+#include <error.h>
+#include <stdlib.h>
+#include <string.h>
+#include <unistd.h>
+
+#include <gnuastro/eps.h>
+#include <gnuastro/pdf.h>
+
+#include <gnuastro-internal/checkset.h>
+
+
+
+
+/*************************************************************
+ ************** Acceptable PDF names ***************
+ *************************************************************/
+int
+gal_pdf_name_is_pdf(char *name)
+{
+ size_t len;
+ len=strlen(name);
+ if (strcmp(&name[len-3], "pdf") == 0
+ || strcmp(&name[len-3], "PDF") == 0)
+ return 1;
+ else
+ return 0;
+}
+
+
+
+
+
+int
+gal_pdf_suffix_is_pdf(char *name)
+{
+ if (strcmp(name, "pdf") == 0 || strcmp(name, ".pdf") == 0
+ || strcmp(name, "PDF") == 0 || strcmp(name, ".PDF") == 0)
+ return 1;
+ else
+ return 0;
+}
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+/*************************************************************
+ ************** Write a PDF image ***************
+ *************************************************************/
+void
+gal_pdf_write(gal_data_t *in, char *filename, float widthincm,
+ uint32_t borderwidth)
+{
+ char command[20000];
+ size_t w_h_in_pt[2];
+ char *epsname=gal_checkset_malloc_cat(filename, ".ps");
+
+ /* Write the EPS file. */
+ gal_eps_write(in, epsname, widthincm, borderwidth, 0, 1);
+
+ /* Get the size of the image in `pt' units. */
+ gal_eps_to_pt(widthincm, in->dsize, w_h_in_pt);
+
+ /* Write the ghostscript command to compile the EPS file to PDF. */
+ sprintf(command, "gs -q -o %s -sDEVICE=pdfwrite -dDEVICEWIDTHPOINTS=%zu"
+ " -dDEVICEHEIGHTPOINTS=%zu -dPDFFitPage %s", filename,
+ w_h_in_pt[0]+2*borderwidth, w_h_in_pt[1]+2*borderwidth,
+ epsname);
+
+ /* Run Ghostscript. */
+ if(system(command))
+ error(EXIT_FAILURE, 0, "the command to convert a PostScript file to "
+ "PDF (`%s') was not successful! The PostScript file (%s) is "
+ "left if you want to convert or use it through any other "
+ "means", command, epsname);
+
+ /* Delete the EPS file. */
+ errno=0;
+ if(unlink(epsname))
+ error(EXIT_FAILURE, errno, "%s", epsname);
+
+ /* Clean up. */
+ free(epsname);
+}
diff --git a/lib/statistics.c b/lib/statistics.c
index 6c19f36..8d2673c 100644
--- a/lib/statistics.c
+++ b/lib/statistics.c
@@ -54,13 +54,12 @@ along with Gnuastro. If not, see
<http://www.gnu.org/licenses/>.
******** Simple statistics *******
****************************************************************/
/* Return the number of non-blank elements in an array as a single element,
- uint64 type data structure. */
+ `size_t' type data structure. */
gal_data_t *
gal_statistics_number(gal_data_t *input)
{
- size_t dsize=1;
- uint64_t counter=0;
- gal_data_t *out=gal_data_alloc(NULL, GAL_TYPE_UINT64, 1, &dsize,
+ size_t counter=0, dsize=1;
+ gal_data_t *out=gal_data_alloc(NULL, GAL_TYPE_SIZE_T, 1, &dsize,
NULL, 1, -1, NULL, NULL, NULL);
/* If there is no blank values in the input, then the total number is
@@ -71,7 +70,7 @@ gal_statistics_number(gal_data_t *input)
counter = input->size;
/* Write the value into memory. */
- *((uint64_t *)(out->array)) = counter;
+ *((size_t *)(out->array)) = counter;
return out;
}
@@ -88,11 +87,16 @@ gal_statistics_minimum(gal_data_t *input)
gal_data_t *out=gal_data_alloc(NULL, gal_tile_block(input)->type, 1,
&dsize, NULL, 1, -1, NULL, NULL, NULL);
- /* Initialize the output with the maximum possible value. */
- gal_type_max(out->type, out->array);
+ /* See if the input actually has any elements. */
+ if(input->size)
+ {
+ /* Initialize the output with the maximum possible value. */
+ gal_type_max(out->type, out->array);
- /* Parse the full input. */
- GAL_TILE_PARSE_OPERATE(input, out, 0, 1, {*o = *i < *o ? *i : *o; ++n;});
+ /* Parse the full input. */
+ GAL_TILE_PARSE_OPERATE( input, out, 0, 1,
+ {*o = *i < *o ? *i : *o; ++n;} );
+ }
/* If there were no usable elements, set the output to blank, then
return. */
@@ -112,12 +116,16 @@ gal_statistics_maximum(gal_data_t *input)
size_t dsize=1, n=0;
gal_data_t *out=gal_data_alloc(NULL, gal_tile_block(input)->type, 1,
&dsize, NULL, 1, -1, NULL, NULL, NULL);
+ /* See if the input actually has any elements. */
+ if(input->size)
+ {
+ /* Initialize the output with the minimum possible value. */
+ gal_type_min(out->type, out->array);
- /* Initialize the output with the minimum possible value. */
- gal_type_min(out->type, out->array);
-
- /* Parse the full input. */
- GAL_TILE_PARSE_OPERATE(input, out, 0, 1, {*o = *i > *o ? *i : *o; ++n;});
+ /* Parse the full input. */
+ GAL_TILE_PARSE_OPERATE(input, out, 0, 1,
+ {*o = *i > *o ? *i : *o; ++n;});
+ }
/* If there were no usable elements, set the output to blank, then
return. */
@@ -138,9 +146,11 @@ gal_statistics_sum(gal_data_t *input)
gal_data_t *out=gal_data_alloc(NULL, GAL_TYPE_FLOAT64, 1, &dsize,
NULL, 1, -1, NULL, NULL, NULL);
- /* Parse the dataset. Note that in `gal_data_alloc' we set the `clear'
- flag to 1, so it will be 0.0f. */
- GAL_TILE_PARSE_OPERATE(input, out, 0, 1, {++n; *o += *i;});
+ /* See if the input actually has any elements. */
+ if(input->size)
+ /* Parse the dataset. Note that in `gal_data_alloc' we set the `clear'
+ flag to 1, so it will be 0.0f. */
+ GAL_TILE_PARSE_OPERATE(input, out, 0, 1, {++n; *o += *i;});
/* If there were no usable elements, set the output to blank, then
return. */
@@ -161,9 +171,11 @@ gal_statistics_mean(gal_data_t *input)
gal_data_t *out=gal_data_alloc(NULL, GAL_TYPE_FLOAT64, 1, &dsize,
NULL, 1, -1, NULL, NULL, NULL);
- /* Parse the dataset. Note that in `gal_data_alloc' we set the `clear'
- flag to 1, so it will be 0.0f. */
- GAL_TILE_PARSE_OPERATE(input, out, 0, 1, {++n; *o += *i;});
+ /* See if the input actually has any elements. */
+ if(input->size)
+ /* Parse the dataset. Note that in `gal_data_alloc' we set the `clear'
+ flag to 1, so it will be 0.0f. */
+ GAL_TILE_PARSE_OPERATE(input, out, 0, 1, {++n; *o += *i;});
/* Above, we calculated the sum and number, so if there were any elements
in the dataset (`n!=0'), divide the sum by the number, otherwise, put
@@ -187,8 +199,10 @@ gal_statistics_std(gal_data_t *input)
gal_data_t *out=gal_data_alloc(NULL, GAL_TYPE_FLOAT64, 1, &dsize,
NULL, 1, -1, NULL, NULL, NULL);
- /* Parse the input. */
- GAL_TILE_PARSE_OPERATE(input, out, 0, 1, {++n; s += *i; s2 += *i * *i;});
+ /* See if the input actually has any elements. */
+ if(input->size)
+ /* Parse the input. */
+ GAL_TILE_PARSE_OPERATE(input, out, 0, 1, {++n; s += *i; s2 += *i * *i;});
/* Write the standard deviation into the output. */
*((double *)(out->array)) = n ? sqrt( (s2-s*s/n)/n ) : GAL_BLANK_FLOAT64;
@@ -209,9 +223,10 @@ gal_statistics_mean_std(gal_data_t *input)
double s=0.0f, s2=0.0f;
gal_data_t *out=gal_data_alloc(NULL, GAL_TYPE_FLOAT64, 1, &dsize,
NULL, 1, -1, NULL, NULL, NULL);
-
- /* Parse the input. */
- GAL_TILE_PARSE_OPERATE(input, out, 0, 1, {++n; s += *i; s2 += *i * *i;});
+ /* See if the input actually has any elements. */
+ if(input->size)
+ /* Parse the input. */
+ GAL_TILE_PARSE_OPERATE(input, out, 0, 1, {++n; s += *i; s2 += *i * *i;});
/* Write in the output values and return. */
((double *)(out->array))[0] = n ? s/n : GAL_BLANK_FLOAT64;
@@ -235,22 +250,26 @@ statistics_median_in_sorted_no_blank(gal_data_t *sorted,
void *median)
{
size_t n=sorted->size;
- switch(sorted->type)
- {
- case GAL_TYPE_UINT8: MED_IN_SORTED( uint8_t ); break;
- case GAL_TYPE_INT8: MED_IN_SORTED( int8_t ); break;
- case GAL_TYPE_UINT16: MED_IN_SORTED( uint16_t ); break;
- case GAL_TYPE_INT16: MED_IN_SORTED( int16_t ); break;
- case GAL_TYPE_UINT32: MED_IN_SORTED( uint32_t ); break;
- case GAL_TYPE_INT32: MED_IN_SORTED( int32_t ); break;
- case GAL_TYPE_UINT64: MED_IN_SORTED( uint64_t ); break;
- case GAL_TYPE_INT64: MED_IN_SORTED( int64_t ); break;
- case GAL_TYPE_FLOAT32: MED_IN_SORTED( float ); break;
- case GAL_TYPE_FLOAT64: MED_IN_SORTED( double ); break;
- default:
- error(EXIT_FAILURE, 0, "%s: type code %d not recognized",
- __func__, sorted->type);
- }
+ /* Do the processing if there are actually any elements. */
+ if(sorted->size)
+ switch(sorted->type)
+ {
+ case GAL_TYPE_UINT8: MED_IN_SORTED( uint8_t ); break;
+ case GAL_TYPE_INT8: MED_IN_SORTED( int8_t ); break;
+ case GAL_TYPE_UINT16: MED_IN_SORTED( uint16_t ); break;
+ case GAL_TYPE_INT16: MED_IN_SORTED( int16_t ); break;
+ case GAL_TYPE_UINT32: MED_IN_SORTED( uint32_t ); break;
+ case GAL_TYPE_INT32: MED_IN_SORTED( int32_t ); break;
+ case GAL_TYPE_UINT64: MED_IN_SORTED( uint64_t ); break;
+ case GAL_TYPE_INT64: MED_IN_SORTED( int64_t ); break;
+ case GAL_TYPE_FLOAT32: MED_IN_SORTED( float ); break;
+ case GAL_TYPE_FLOAT64: MED_IN_SORTED( double ); break;
+ default:
+ error(EXIT_FAILURE, 0, "%s: type code %d not recognized",
+ __func__, sorted->type);
+ }
+ else
+ gal_blank_write(median, sorted->type);
}
@@ -270,7 +289,10 @@ gal_statistics_median(gal_data_t *input, int inplace)
NULL, NULL, NULL);
/* Write the median. */
- statistics_median_in_sorted_no_blank(nbs, out->array);
+ if(nbs->size)
+ statistics_median_in_sorted_no_blank(nbs, out->array);
+ else
+ gal_blank_write(out->array, out->type);
/* Clean up (if necessary), then return the output */
if(nbs!=input) gal_data_free(nbs);
@@ -327,19 +349,26 @@ gal_statistics_quantile(gal_data_t *input, double
quantile, int inplace)
gal_data_t *out=gal_data_alloc(NULL, nbs->type, 1, &dsize,
NULL, 1, -1, NULL, NULL, NULL);
- /* Find the index of the quantile. */
- index=gal_statistics_quantile_index(nbs->size, quantile);
-
- /* Write the value at this index into the output. */
- if(index==GAL_BLANK_SIZE_T)
+ /* Only continue processing if there are non-blank elements. */
+ if(nbs->size)
{
- blank=gal_data_malloc_array(nbs->type, 1, __func__, "blank");
- memcpy(out->array, blank, gal_type_sizeof(nbs->type));
- free(blank);
+ /* Find the index of the quantile. */
+ index=gal_statistics_quantile_index(nbs->size, quantile);
+
+ /* Write the value at this index into the output. */
+ if(index==GAL_BLANK_SIZE_T)
+ {
+ blank=gal_data_malloc_array(nbs->type, 1, __func__, "blank");
+ memcpy(out->array, blank, gal_type_sizeof(nbs->type));
+ free(blank);
+ }
+ else
+ memcpy(out->array,
+ gal_data_ptr_increment(nbs->array, index, nbs->type),
+ gal_type_sizeof(nbs->type));
}
else
- memcpy(out->array, gal_data_ptr_increment(nbs->array, index, nbs->type),
- gal_type_sizeof(nbs->type));
+ gal_blank_write(out->array, out->type);
/* Clean up and return. */
if(nbs!=input) gal_data_free(nbs);
@@ -398,22 +427,30 @@ gal_statistics_quantile_function_index(gal_data_t *input,
gal_data_t *value,
error(EXIT_FAILURE, 0, "%s: the types of the input dataset and requested "
"value have to be the same", __func__);
- /* Find the result: */
- switch(nbs->type)
+ /* Only continue processing if we have non-blank elements. */
+ if(nbs->size)
+ /* Find the result: */
+ switch(nbs->type)
+ {
+ case GAL_TYPE_UINT8: STATS_QFUNC_IND( uint8_t ); break;
+ case GAL_TYPE_INT8: STATS_QFUNC_IND( int8_t ); break;
+ case GAL_TYPE_UINT16: STATS_QFUNC_IND( uint16_t ); break;
+ case GAL_TYPE_INT16: STATS_QFUNC_IND( int16_t ); break;
+ case GAL_TYPE_UINT32: STATS_QFUNC_IND( uint32_t ); break;
+ case GAL_TYPE_INT32: STATS_QFUNC_IND( int32_t ); break;
+ case GAL_TYPE_UINT64: STATS_QFUNC_IND( uint64_t ); break;
+ case GAL_TYPE_INT64: STATS_QFUNC_IND( int64_t ); break;
+ case GAL_TYPE_FLOAT32: STATS_QFUNC_IND( float ); break;
+ case GAL_TYPE_FLOAT64: STATS_QFUNC_IND( double ); break;
+ default:
+ error(EXIT_FAILURE, 0, "%s: type code %d not recognized",
+ __func__, nbs->type);
+ }
+ else
{
- case GAL_TYPE_UINT8: STATS_QFUNC_IND( uint8_t ); break;
- case GAL_TYPE_INT8: STATS_QFUNC_IND( int8_t ); break;
- case GAL_TYPE_UINT16: STATS_QFUNC_IND( uint16_t ); break;
- case GAL_TYPE_INT16: STATS_QFUNC_IND( int16_t ); break;
- case GAL_TYPE_UINT32: STATS_QFUNC_IND( uint32_t ); break;
- case GAL_TYPE_INT32: STATS_QFUNC_IND( int32_t ); break;
- case GAL_TYPE_UINT64: STATS_QFUNC_IND( uint64_t ); break;
- case GAL_TYPE_INT64: STATS_QFUNC_IND( int64_t ); break;
- case GAL_TYPE_FLOAT32: STATS_QFUNC_IND( float ); break;
- case GAL_TYPE_FLOAT64: STATS_QFUNC_IND( double ); break;
- default:
- error(EXIT_FAILURE, 0, "%s: type code %d not recognized",
- __func__, nbs->type);
+ error(0, 0, "%s: no non-blank elements. The quantile function is not "
+ "defined for a zero-sized array\n", __func__);
+ index=GAL_BLANK_SIZE_T;
}
/* Clean up and return. */
@@ -448,32 +485,38 @@ gal_statistics_quantile_function(gal_data_t *input,
gal_data_t *value,
NULL, 1, -1, NULL, NULL, NULL);
size_t ind=gal_statistics_quantile_function_index(input, value, inplace);
- /* Note that counting of the index starts from 0, so for the quantile we
- should divided by (size - 1). */
- d=out->array;
- if(ind==GAL_BLANK_SIZE_T)
+ /* Only continue processing if there are non-blank values. */
+ if(nbs->size)
{
- /* See if the value is larger or smaller than the input's minimum or
- maximum. */
- switch(nbs->type)
+ /* Note that counting of the index starts from 0, so for the quantile
+ we should divided by (size - 1). */
+ d=out->array;
+ if(ind==GAL_BLANK_SIZE_T)
{
- case GAL_TYPE_UINT8: STATS_QFUNC( uint8_t ); break;
- case GAL_TYPE_INT8: STATS_QFUNC( int8_t ); break;
- case GAL_TYPE_UINT16: STATS_QFUNC( uint16_t ); break;
- case GAL_TYPE_INT16: STATS_QFUNC( int16_t ); break;
- case GAL_TYPE_UINT32: STATS_QFUNC( uint32_t ); break;
- case GAL_TYPE_INT32: STATS_QFUNC( int32_t ); break;
- case GAL_TYPE_UINT64: STATS_QFUNC( uint64_t ); break;
- case GAL_TYPE_INT64: STATS_QFUNC( int64_t ); break;
- case GAL_TYPE_FLOAT32: STATS_QFUNC( float ); break;
- case GAL_TYPE_FLOAT64: STATS_QFUNC( double ); break;
- default:
- error(EXIT_FAILURE, 0, "%s: type code %d not recognized",
- __func__, nbs->type);
+ /* See if the value is larger or smaller than the input's minimum
+ or maximum. */
+ switch(nbs->type)
+ {
+ case GAL_TYPE_UINT8: STATS_QFUNC( uint8_t ); break;
+ case GAL_TYPE_INT8: STATS_QFUNC( int8_t ); break;
+ case GAL_TYPE_UINT16: STATS_QFUNC( uint16_t ); break;
+ case GAL_TYPE_INT16: STATS_QFUNC( int16_t ); break;
+ case GAL_TYPE_UINT32: STATS_QFUNC( uint32_t ); break;
+ case GAL_TYPE_INT32: STATS_QFUNC( int32_t ); break;
+ case GAL_TYPE_UINT64: STATS_QFUNC( uint64_t ); break;
+ case GAL_TYPE_INT64: STATS_QFUNC( int64_t ); break;
+ case GAL_TYPE_FLOAT32: STATS_QFUNC( float ); break;
+ case GAL_TYPE_FLOAT64: STATS_QFUNC( double ); break;
+ default:
+ error(EXIT_FAILURE, 0, "%s: type code %d not recognized",
+ __func__, nbs->type);
+ }
}
+ else
+ d[0] = (double)ind / ((double)(nbs->size - 1));
}
else
- d[0] = (double)ind / ((double)(nbs->size - 1));
+ gal_blank_write(out->array, out->type);
/* Clean up and return. */
if(nbs!=input) gal_data_free(nbs);
@@ -1056,6 +1099,8 @@ gal_statistics_mode_mirror_plots(gal_data_t *input,
gal_data_t *value,
gal_data_t *nbs=gal_statistics_no_blank_sorted(input, inplace);
size_t ind=gal_statistics_quantile_function_index(nbs, value, inplace);
+ /* Only continue if we actually have non-blank elements. */
+ if(nbs->size==0) return NULL;
/* If the given mirror was outside the range of the input, then index
will be 0 (below the range) or -1 (above the range), in that case, we
@@ -1130,31 +1175,36 @@ gal_statistics_is_sorted(gal_data_t *input)
{
/* A one-element dataset can be considered, sorted, so we'll just return
1 (for sorted and increasing). */
- if(input->size==1) return GAL_STATISTICS_SORTED_INCREASING;
-
- /* Do the check. */
- switch(input->type)
+ switch(input->size)
{
- case GAL_TYPE_UINT8: IS_SORTED( uint8_t ); break;
- case GAL_TYPE_INT8: IS_SORTED( int8_t ); break;
- case GAL_TYPE_UINT16: IS_SORTED( uint16_t ); break;
- case GAL_TYPE_INT16: IS_SORTED( int16_t ); break;
- case GAL_TYPE_UINT32: IS_SORTED( uint32_t ); break;
- case GAL_TYPE_INT32: IS_SORTED( int32_t ); break;
- case GAL_TYPE_UINT64: IS_SORTED( uint64_t ); break;
- case GAL_TYPE_INT64: IS_SORTED( int64_t ); break;
- case GAL_TYPE_FLOAT32: IS_SORTED( float ); break;
- case GAL_TYPE_FLOAT64: IS_SORTED( double ); break;
+ case 0: return GAL_STATISTICS_SORTED_INVALID;
+ case 1: return GAL_STATISTICS_SORTED_INCREASING;
+
+ /* Do the check. */
default:
- error(EXIT_FAILURE, 0, "%s: type code %d not recognized",
- __func__, input->type);
+ switch(input->type)
+ {
+ case GAL_TYPE_UINT8: IS_SORTED( uint8_t ); break;
+ case GAL_TYPE_INT8: IS_SORTED( int8_t ); break;
+ case GAL_TYPE_UINT16: IS_SORTED( uint16_t ); break;
+ case GAL_TYPE_INT16: IS_SORTED( int16_t ); break;
+ case GAL_TYPE_UINT32: IS_SORTED( uint32_t ); break;
+ case GAL_TYPE_INT32: IS_SORTED( int32_t ); break;
+ case GAL_TYPE_UINT64: IS_SORTED( uint64_t ); break;
+ case GAL_TYPE_INT64: IS_SORTED( int64_t ); break;
+ case GAL_TYPE_FLOAT32: IS_SORTED( float ); break;
+ case GAL_TYPE_FLOAT64: IS_SORTED( double ); break;
+ default:
+ error(EXIT_FAILURE, 0, "%s: type code %d not recognized",
+ __func__, input->type);
+ }
}
/* Control shouldn't reach this point. */
- error(EXIT_FAILURE, 0, "%s: a bug! Please contact us at %s so we can fix the
"
- "problem. Control must not have reached the end of this function",
- __func__, PACKAGE_BUGREPORT);
- return -1;
+ error(EXIT_FAILURE, 0, "%s: a bug! Please contact us at %s so we can "
+ "fix the problem. Control must not have reached the end of this "
+ "function", __func__, PACKAGE_BUGREPORT);
+ return GAL_STATISTICS_SORTED_INVALID;
}
@@ -1169,32 +1219,33 @@ gal_statistics_is_sorted(gal_data_t *input)
void
gal_statistics_sort_increasing(gal_data_t *input)
{
- switch(input->type)
- {
- case GAL_TYPE_UINT8:
- STATISTICS_SORT(gal_qsort_uint8_increasing); break;
- case GAL_TYPE_INT8:
- STATISTICS_SORT(gal_qsort_int8_increasing); break;
- case GAL_TYPE_UINT16:
- STATISTICS_SORT(gal_qsort_uint16_increasing); break;
- case GAL_TYPE_INT16:
- STATISTICS_SORT(gal_qsort_int16_increasing); break;
- case GAL_TYPE_UINT32:
- STATISTICS_SORT(gal_qsort_uint32_increasing); break;
- case GAL_TYPE_INT32:
- STATISTICS_SORT(gal_qsort_int32_increasing); break;
- case GAL_TYPE_UINT64:
- STATISTICS_SORT(gal_qsort_uint64_increasing); break;
- case GAL_TYPE_INT64:
- STATISTICS_SORT(gal_qsort_int64_increasing); break;
- case GAL_TYPE_FLOAT32:
- STATISTICS_SORT(gal_qsort_float32_increasing); break;
- case GAL_TYPE_FLOAT64:
- STATISTICS_SORT(gal_qsort_float64_increasing); break;
- default:
- error(EXIT_FAILURE, 0, "%s: type code %d not recognized",
- __func__, input->type);
- }
+ if(input->size)
+ switch(input->type)
+ {
+ case GAL_TYPE_UINT8:
+ STATISTICS_SORT(gal_qsort_uint8_increasing); break;
+ case GAL_TYPE_INT8:
+ STATISTICS_SORT(gal_qsort_int8_increasing); break;
+ case GAL_TYPE_UINT16:
+ STATISTICS_SORT(gal_qsort_uint16_increasing); break;
+ case GAL_TYPE_INT16:
+ STATISTICS_SORT(gal_qsort_int16_increasing); break;
+ case GAL_TYPE_UINT32:
+ STATISTICS_SORT(gal_qsort_uint32_increasing); break;
+ case GAL_TYPE_INT32:
+ STATISTICS_SORT(gal_qsort_int32_increasing); break;
+ case GAL_TYPE_UINT64:
+ STATISTICS_SORT(gal_qsort_uint64_increasing); break;
+ case GAL_TYPE_INT64:
+ STATISTICS_SORT(gal_qsort_int64_increasing); break;
+ case GAL_TYPE_FLOAT32:
+ STATISTICS_SORT(gal_qsort_float32_increasing); break;
+ case GAL_TYPE_FLOAT64:
+ STATISTICS_SORT(gal_qsort_float64_increasing); break;
+ default:
+ error(EXIT_FAILURE, 0, "%s: type code %d not recognized",
+ __func__, input->type);
+ }
}
@@ -1205,114 +1256,128 @@ gal_statistics_sort_increasing(gal_data_t *input)
void
gal_statistics_sort_decreasing(gal_data_t *input)
{
- switch(input->type)
- {
- case GAL_TYPE_UINT8:
- STATISTICS_SORT(gal_qsort_uint8_decreasing); break;
- case GAL_TYPE_INT8:
- STATISTICS_SORT(gal_qsort_int8_decreasing); break;
- case GAL_TYPE_UINT16:
- STATISTICS_SORT(gal_qsort_uint16_decreasing); break;
- case GAL_TYPE_INT16:
- STATISTICS_SORT(gal_qsort_int16_decreasing); break;
- case GAL_TYPE_UINT32:
- STATISTICS_SORT(gal_qsort_uint32_decreasing); break;
- case GAL_TYPE_INT32:
- STATISTICS_SORT(gal_qsort_int32_decreasing); break;
- case GAL_TYPE_UINT64:
- STATISTICS_SORT(gal_qsort_uint64_decreasing); break;
- case GAL_TYPE_INT64:
- STATISTICS_SORT(gal_qsort_int64_decreasing); break;
- case GAL_TYPE_FLOAT32:
- STATISTICS_SORT(gal_qsort_float32_decreasing); break;
- case GAL_TYPE_FLOAT64:
- STATISTICS_SORT(gal_qsort_float64_decreasing); break;
- default:
- error(EXIT_FAILURE, 0, "%s: type code %d not recognized",
- __func__, input->type);
- }
+ if(input->size)
+ switch(input->type)
+ {
+ case GAL_TYPE_UINT8:
+ STATISTICS_SORT(gal_qsort_uint8_decreasing); break;
+ case GAL_TYPE_INT8:
+ STATISTICS_SORT(gal_qsort_int8_decreasing); break;
+ case GAL_TYPE_UINT16:
+ STATISTICS_SORT(gal_qsort_uint16_decreasing); break;
+ case GAL_TYPE_INT16:
+ STATISTICS_SORT(gal_qsort_int16_decreasing); break;
+ case GAL_TYPE_UINT32:
+ STATISTICS_SORT(gal_qsort_uint32_decreasing); break;
+ case GAL_TYPE_INT32:
+ STATISTICS_SORT(gal_qsort_int32_decreasing); break;
+ case GAL_TYPE_UINT64:
+ STATISTICS_SORT(gal_qsort_uint64_decreasing); break;
+ case GAL_TYPE_INT64:
+ STATISTICS_SORT(gal_qsort_int64_decreasing); break;
+ case GAL_TYPE_FLOAT32:
+ STATISTICS_SORT(gal_qsort_float32_decreasing); break;
+ case GAL_TYPE_FLOAT64:
+ STATISTICS_SORT(gal_qsort_float64_decreasing); break;
+ default:
+ error(EXIT_FAILURE, 0, "%s: type code %d not recognized",
+ __func__, input->type);
+ }
}
-/* Return a dataset that has doesn't have blank values and is sorted. If
- the `inplace' value is set to 1, then the input array will be modified,
+/* Return a dataset that doesn't have blank values and is sorted. If the
+ `inplace' value is set to 1, then the input array will be modified,
otherwise, a new array will be allocated with the desired properties. So
if it is already sorted and has blank values, the `inplace' variable is
irrelevant.
This function can also work on tiles, in that case, `inplace' is
useless, because a tile doesn't own its dataset and the dataset is not
- contiguous.
-*/
+ contiguous. */
gal_data_t *
gal_statistics_no_blank_sorted(gal_data_t *input, int inplace)
{
int sortstatus;
gal_data_t *contig, *noblank, *sorted;
-
- /* If this is a tile, then first we have to copy it into a contiguous
- piece of memory. After this step, we will only be dealing with
- `contig' (for a contiguous patch of memory). */
- if(input->block)
+ /* We need to account for the case that there are no elements in the
+ input. */
+ if(input->size)
{
- /* Copy the input into a contiguous patch of memory. */
- contig=gal_data_copy(input);
+ /* If this is a tile, then first we have to copy it into a contiguous
+ piece of memory. After this step, we will only be dealing with
+ `contig' (for a contiguous patch of memory). */
+ if(input->block)
+ {
+ /* Copy the input into a contiguous patch of memory. */
+ contig=gal_data_copy(input);
- /* When the data was a tile, we have already copied the array into a
- separate allocated space. So to avoid any further copying, we will
- just set the `inplace' variable to 1. */
- inplace=1;
- }
- else contig=input;
+ /* When the data was a tile, we have already copied the array
+ into a separate allocated space. So to avoid any further
+ copying, we will just set the `inplace' variable to 1. */
+ inplace=1;
+ }
+ else contig=input;
- /* Make sure there is no blanks in the array that will be used. After
- this step, we won't be dealing with `input' any more, but with
- `noblank'. */
- if( gal_blank_present(contig, inplace) )
- {
- /* See if we should allocate a new dataset to remove blanks or if we
- can use the actual contiguous patch of memory. */
- noblank = inplace ? contig : gal_data_copy(contig);
- gal_blank_remove(noblank);
-
- /* If we are working in place, then mark that there are no blank
- pixels. */
- if(inplace)
+ /* Make sure there is no blanks in the array that will be used. After
+ this step, we won't be dealing with `input' any more, but with
+ `noblank'. */
+ if( gal_blank_present(contig, inplace) )
{
- noblank->flag |= GAL_DATA_FLAG_BLANK_CH;
- noblank->flag &= ~GAL_DATA_FLAG_HASBLANK;
+ /* See if we should allocate a new dataset to remove blanks or if
+ we can use the actual contiguous patch of memory. */
+ noblank = inplace ? contig : gal_data_copy(contig);
+ gal_blank_remove(noblank);
+
+ /* If we are working in place, then mark that there are no blank
+ pixels. */
+ if(inplace)
+ {
+ noblank->flag |= GAL_DATA_FLAG_BLANK_CH;
+ noblank->flag &= ~GAL_DATA_FLAG_HASBLANK;
+ }
}
- }
- else noblank=contig;
+ else noblank=contig;
- /* Make sure the array is sorted. After this step, we won't be dealing
- with `noblank' any more but with `sorted'. */
- sortstatus=gal_statistics_is_sorted(noblank);
- if( sortstatus )
- {
- sorted=noblank;
- sorted->status=sortstatus;
- }
- else
- {
- if(inplace) sorted=noblank;
- else
+ /* If there are any non-blank elements, make sure the array is
+ sorted. After this step, we won't be dealing with `noblank' any
+ more but with `sorted'. */
+ if(noblank->size)
{
- if(noblank!=input) /* no-blank has already been allocated. */
- sorted=noblank;
+ sortstatus=gal_statistics_is_sorted(noblank);
+ if( sortstatus )
+ {
+ sorted=noblank;
+ sorted->status=sortstatus;
+ }
else
- sorted=gal_data_copy(noblank);
+ {
+ if(inplace) sorted=noblank;
+ else
+ {
+ if(noblank!=input) /* no-blank is already allocated. */
+ sorted=noblank;
+ else
+ sorted=gal_data_copy(noblank);
+ }
+ gal_statistics_sort_increasing(sorted);
+ sorted->status=GAL_STATISTICS_SORTED_INCREASING;
+ }
}
- gal_statistics_sort_increasing(sorted);
- sorted->status=GAL_STATISTICS_SORTED_INCREASING;
+ else
+ sorted=noblank;
}
+ /* When the input's size is zero, just return the actual input. */
+ else
+ sorted=input;
+
/* Return final array. */
return sorted;
@@ -1388,6 +1453,8 @@ gal_statistics_regular_bins(gal_data_t *input, gal_data_t
*inrange,
if(numbins==0)
error(EXIT_FAILURE, 0, "%s: `numbins' cannot be given a value of 0",
__func__);
+ if(input->size==0)
+ error(EXIT_FAILURE, 0, "%s: input's size is 0", __func__);
/* Set the minimum and maximum values. */
@@ -1409,8 +1476,8 @@ gal_statistics_regular_bins(gal_data_t *input, gal_data_t
*inrange,
/* If the minimum isn't set (is blank), find it. */
if( isnan(ra[0]) )
{
- tmp=gal_data_copy_to_new_type_free(gal_statistics_minimum(input),
- GAL_TYPE_FLOAT64);
+ tmp=gal_data_copy_to_new_type_free(
+ gal_statistics_minimum(input), GAL_TYPE_FLOAT64);
min=*((double *)(tmp->array));
gal_data_free(tmp);
}
@@ -1520,6 +1587,8 @@ gal_statistics_histogram(gal_data_t *input, gal_data_t
*bins, int normalize,
if(bins->status!=GAL_STATISTICS_BINS_REGULAR)
error(EXIT_FAILURE, 0, "%s: the input bins are not regular. Currently "
"it is only implemented for regular bins", __func__);
+ if(input->size==0)
+ error(EXIT_FAILURE, 0, "%s: input's size is 0", __func__);
/* Check if normalize and `maxone' are not called together. */
@@ -1646,6 +1715,8 @@ gal_statistics_cfp(gal_data_t *input, gal_data_t *bins,
int normalize)
if(bins->status!=GAL_STATISTICS_BINS_REGULAR)
error(EXIT_FAILURE, 0, "%s: the input bins are not regular. Currently "
"it is only implemented for regular bins", __func__);
+ if(input->size==0)
+ error(EXIT_FAILURE, 0, "%s: input's size is 0", __func__);
/* Prepare the histogram. */
@@ -1800,6 +1871,7 @@ gal_data_t *
gal_statistics_sigma_clip(gal_data_t *input, float multip, float param,
int inplace, int quiet)
{
+ float *oa;
void *start, *nbs_array;
double *med, *mean, *std;
uint8_t bytolerance = param>=1.0f ? 0 : 1;
@@ -1831,99 +1903,110 @@ gal_statistics_sigma_clip(gal_data_t *input, float
multip, float param,
NULL, NULL, NULL);
- /* Print the comments. */
- if(!quiet)
- printf("%-8s %-10s %-15s %-15s %-15s\n",
- "round", "number", "median", "mean", "STD");
-
-
- /* Do the clipping, but first initialize the values that will be changed
- during the clipping: the start of the array and the array's size. */
- size=nbs->size;
- sortstatus=nbs->status;
- nbs_array=start=nbs->array;
- while(num<maxnum)
+ /* Only continue processing if we have non-blank elements. */
+ oa=out->array;
+ nbs_array=nbs->array;
+ if(nbs->size==0)
{
- /* Find the median. */
- statistics_median_in_sorted_no_blank(nbs, median_i->array);
- median_d=gal_data_copy_to_new_type(median_i, GAL_TYPE_FLOAT64);
-
- /* Find the average and Standard deviation, note that both `start'
- and `size' will be different in the next round. */
- nbs->array = start;
- nbs->size = oldsize = size;
- meanstd=gal_statistics_mean_std(nbs);
-
- /* Put the three final values in usable (with a type) pointers. */
- med = median_d->array;
- mean = meanstd->array;
- std = &((double *)(meanstd->array))[1];
-
- /* If the user wanted to view the steps, show it to them. */
if(!quiet)
- printf("%-8zu %-10zu %-15g %-15g %-15g\n",
- num+1, size, *med, *mean, *std);
-
- /* If we are to work by tolerance, then check if we should jump out
- of the loop. Normally, `oldstd' should be larger than std, because
- the possible outliers have been removed. If it is not, it means
- that we have clipped too much and must stop anyway, so we don't
- need an absolute value on the difference! */
- if( bytolerance && num>0 && ((oldstd - *std) / *std) < param )
- break;
-
- /* Clip all the elements outside of the desired range: since the
- array is sorted, this means to just change the starting pointer
- and size of the array. */
- switch(type)
+ printf("NO SIGMA-CLIPPING: all input elements are blank or input's "
+ "size is zero.\n");
+ oa[0] = oa[1] = oa[2] = oa[3] = NAN;
+ }
+ else
+ {
+ /* Print the comments. */
+ if(!quiet)
+ printf("%-8s %-10s %-15s %-15s %-15s\n",
+ "round", "number", "median", "mean", "STD");
+
+
+ /* Do the clipping, but first initialize the values that will be
+ changed during the clipping: the start of the array and the
+ array's size. */
+ size=nbs->size;
+ start=nbs->array;
+ sortstatus=nbs->status;
+ while(num<maxnum && size)
{
- case GAL_TYPE_UINT8: SIGCLIP( uint8_t ); break;
- case GAL_TYPE_INT8: SIGCLIP( int8_t ); break;
- case GAL_TYPE_UINT16: SIGCLIP( uint16_t ); break;
- case GAL_TYPE_INT16: SIGCLIP( int16_t ); break;
- case GAL_TYPE_UINT32: SIGCLIP( uint32_t ); break;
- case GAL_TYPE_INT32: SIGCLIP( int32_t ); break;
- case GAL_TYPE_UINT64: SIGCLIP( uint64_t ); break;
- case GAL_TYPE_INT64: SIGCLIP( int64_t ); break;
- case GAL_TYPE_FLOAT32: SIGCLIP( float ); break;
- case GAL_TYPE_FLOAT64: SIGCLIP( double ); break;
- default:
- error(EXIT_FAILURE, 0, "%s: type code %d not recognized",
- __func__, type);
- }
+ /* Find the median. */
+ statistics_median_in_sorted_no_blank(nbs, median_i->array);
+ median_d=gal_data_copy_to_new_type(median_i, GAL_TYPE_FLOAT64);
+
+ /* Find the average and Standard deviation, note that both
+ `start' and `size' will be different in the next round. */
+ nbs->array = start;
+ nbs->size = oldsize = size;
+ meanstd=gal_statistics_mean_std(nbs);
+
+ /* Put the three final values in usable (with a type)
+ pointers. */
+ med = median_d->array;
+ mean = meanstd->array;
+ std = &((double *)(meanstd->array))[1];
+
+ /* If the user wanted to view the steps, show it to them. */
+ if(!quiet)
+ printf("%-8zu %-10zu %-15g %-15g %-15g\n",
+ num+1, size, *med, *mean, *std);
+
+ /* If we are to work by tolerance, then check if we should jump
+ out of the loop. Normally, `oldstd' should be larger than std,
+ because the possible outliers have been removed. If it is not,
+ it means that we have clipped too much and must stop anyway,
+ so we don't need an absolute value on the difference! */
+ if( bytolerance && num>0 && ((oldstd - *std) / *std) < param )
+ { gal_data_free(meanstd); gal_data_free(median_d); break; }
+
+ /* Clip all the elements outside of the desired range: since the
+ array is sorted, this means to just change the starting
+ pointer and size of the array. */
+ switch(type)
+ {
+ case GAL_TYPE_UINT8: SIGCLIP( uint8_t ); break;
+ case GAL_TYPE_INT8: SIGCLIP( int8_t ); break;
+ case GAL_TYPE_UINT16: SIGCLIP( uint16_t ); break;
+ case GAL_TYPE_INT16: SIGCLIP( int16_t ); break;
+ case GAL_TYPE_UINT32: SIGCLIP( uint32_t ); break;
+ case GAL_TYPE_INT32: SIGCLIP( int32_t ); break;
+ case GAL_TYPE_UINT64: SIGCLIP( uint64_t ); break;
+ case GAL_TYPE_INT64: SIGCLIP( int64_t ); break;
+ case GAL_TYPE_FLOAT32: SIGCLIP( float ); break;
+ case GAL_TYPE_FLOAT64: SIGCLIP( double ); break;
+ default:
+ error(EXIT_FAILURE, 0, "%s: type code %d not recognized",
+ __func__, type);
+ }
- /* Set the values from this round in the old elements, so the next
- round can compare with, and return then if necessary. */
- oldmed = *med;
- oldstd = *std;
- oldmean = *mean;
- ++num;
+ /* Set the values from this round in the old elements, so the
+ next round can compare with, and return then if necessary. */
+ oldmed = *med;
+ oldstd = *std;
+ oldmean = *mean;
+ ++num;
- /* Clean up: */
- gal_data_free(meanstd);
- gal_data_free(median_d);
- }
+ /* Clean up: */
+ gal_data_free(meanstd);
+ gal_data_free(median_d);
+ }
- /* If we were in tolerance mode and `num' and `maxnum' are equal (the
- loop didn't stop by tolerance), so the outputs should be NaN. */
- out->status=num;
- if( bytolerance && num==maxnum )
- {
- ((float *)(out->array))[0] = NAN;
- ((float *)(out->array))[1] = NAN;
- ((float *)(out->array))[2] = NAN;
- ((float *)(out->array))[3] = NAN;
- }
- else
- {
- ((float *)(out->array))[0] = size;
- ((float *)(out->array))[1] = oldmed;
- ((float *)(out->array))[2] = oldmean;
- ((float *)(out->array))[3] = oldstd;
+ /* If we were in tolerance mode and `num' and `maxnum' are equal (the
+ loop didn't stop by tolerance), so the outputs should be NaN. */
+ out->status=num;
+ if( size==0 || (bytolerance && num==maxnum) )
+ oa[0] = oa[1] = oa[2] = oa[3] = NAN;
+ else
+ {
+ oa[0] = size;
+ oa[1] = oldmed;
+ oa[2] = oldmean;
+ oa[3] = oldstd;
+ }
}
/* Clean up and return. */
nbs->array=nbs_array;
+ gal_data_free(median_i);
if(nbs!=input) gal_data_free(nbs);
return out;
}
diff --git a/lib/tiff.c b/lib/tiff.c
new file mode 100644
index 0000000..450e55f
--- /dev/null
+++ b/lib/tiff.c
@@ -0,0 +1,606 @@
+/*********************************************************************
+tiff -- functions to read and write TIFF files.
+This is part of GNU Astronomy Utilities (Gnuastro) package.
+
+Original author:
+ Mohammad Akhlaghi <mohammad@akhlaghi.org>
+Contributing author(s):
+Copyright (C) 2018, Free Software Foundation, Inc.
+
+Gnuastro is free software: you can redistribute it and/or modify it
+under the terms of the GNU General Public License as published by the
+Free Software Foundation, either version 3 of the License, or (at your
+option) any later version.
+
+Gnuastro is distributed in the hope that it will be useful, but
+WITHOUT ANY WARRANTY; without even the implied warranty of
+MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the GNU
+General Public License for more details.
+
+You should have received a copy of the GNU General Public License
+along with Gnuastro. If not, see <http://www.gnu.org/licenses/>.
+**********************************************************************/
+#include <config.h>
+
+#include <stdio.h>
+#include <errno.h>
+#include <error.h>
+#include <stdlib.h>
+#include <string.h>
+#ifdef HAVE_LIBTIFF
+ #include <tiffio.h>
+#endif
+
+#include <gnuastro/fits.h>
+#include <gnuastro/data.h>
+#include <gnuastro/list.h>
+#include <gnuastro/tiff.h>
+
+
+
+
+
+
+
+
+
+
+/*************************************************************
+ ************** Acceptable TIFF names **************
+ *************************************************************/
+int
+gal_tiff_name_is_tiff(char *name)
+{
+ size_t len;
+ len=strlen(name);
+ if ( ( len>=3 && strcmp(&name[len-3], "tif") == 0 )
+ || ( len>=3 && strcmp(&name[len-3], "TIF") == 0 )
+ || ( len>=4 && strcmp(&name[len-4], "tiff") == 0 )
+ || ( len>=4 && strcmp(&name[len-4], "TIFF") == 0 ) )
+ return 1;
+ else
+ return 0;
+}
+
+
+
+
+
+int
+gal_tiff_suffix_is_tiff(char *name)
+{
+ if (strcmp(name, "tif") == 0 || strcmp(name, ".tif") == 0
+ || strcmp(name, "TIF") == 0 || strcmp(name, ".TIF") == 0
+ || strcmp(name, "tiff") == 0 || strcmp(name, ".tiff") == 0
+ || strcmp(name, "TIFF") == 0 || strcmp(name, ".TIFF") == 0 )
+ return 1;
+ else
+ return 0;
+}
+
+
+
+
+
+/* Users may define the TIFF directory to read as a string, in that case,
+ this function can be used to convert it to a `size_t' for use in
+ `gal_tiff_read'. */
+size_t
+gal_tiff_dir_string_read(char *string)
+{
+ long dir;
+ char *tailptr=NULL;
+
+ /* Read the given directory string. */
+ errno=0;
+ dir=strtol(string, &tailptr, 0);
+ if(tailptr==string)
+ error(EXIT_FAILURE, 0, "%s: `%s' couldn't be read as an integer",
+ __func__, string);
+ if(errno)
+ error(EXIT_FAILURE, errno, "%s: reading %s", __func__, string);
+ if(dir<0)
+ error(EXIT_FAILURE, 0, "%s: %ld is a negative integer, it must be "
+ "positive", __func__, dir);
+
+ /* Return the result. */
+ return dir;
+}
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+/*************************************************************
+ ************** Read a JPEG image **************
+ *************************************************************/
+#ifdef HAVE_LIBTIFF
+static void
+tiff_read_tag(TIFF *tif, ttag_t tag, void *out, char *filename, size_t dir)
+{
+ /* Read the tag */
+ if( !TIFFGetField(tif, tag, out) )
+ error(EXIT_FAILURE, 0, "%s: %s (dir %zu): tag %d couldn't be fetched",
+ __func__, filename, dir, tag);
+}
+
+
+
+
+
+/* Convert the TIFF type code into Gnuastro's type code.*/
+static uint8_t
+tiff_type_read(TIFF *tif, char *filename, size_t dir)
+{
+ uint16_t bitspersample, sampleformat;
+
+ /* Read the number of bits and the corresponding type. */
+ tiff_read_tag(tif, TIFFTAG_BITSPERSAMPLE, &bitspersample, filename, dir);
+
+ /* Read the formatting of each pixel. If no such keyword exists, use the
+ value of `SAMPLEFORMAT_UINT'. */
+ if( TIFFGetField(tif, TIFFTAG_SAMPLEFORMAT, &sampleformat) != 1 )
+ sampleformat=SAMPLEFORMAT_UINT;
+
+ /* Read the datatype. */
+ switch(sampleformat)
+ {
+ /* Unsigned integer types. */
+ case SAMPLEFORMAT_UINT:
+ switch(bitspersample)
+ {
+ case 8: return GAL_TYPE_UINT8;
+ case 16: return GAL_TYPE_UINT16;
+ case 32: return GAL_TYPE_UINT32;
+ case 64: return GAL_TYPE_UINT64;
+ default:
+ error(EXIT_FAILURE, 0, "%s: %s (dir %zu): %u-bit samples not "
+ "recognized for UNSIGNED-int format", __func__, filename,
+ dir, bitspersample);
+ }
+ break;
+
+ /* Signed integer types. */
+ case SAMPLEFORMAT_INT:
+ switch(bitspersample)
+ {
+ case 8: return GAL_TYPE_INT8;
+ case 16: return GAL_TYPE_INT16;
+ case 32: return GAL_TYPE_INT32;
+ case 64: return GAL_TYPE_INT64;
+ default:
+ error(EXIT_FAILURE, 0, "%s: %s (dir %zu): %u-bit samples not "
+ "recognized for SIGNED-int format", __func__, filename,
+ dir, bitspersample);
+ }
+ break;
+
+
+ /* Floating point types. */
+ case SAMPLEFORMAT_IEEEFP:
+ switch(bitspersample)
+ {
+ case 32: return GAL_TYPE_FLOAT32;
+ case 64: return GAL_TYPE_FLOAT64;
+ default:
+ error(EXIT_FAILURE, 0, "%s: %s (dir %zu): %u-bit samples not "
+ "recognized for floating point format", __func__, filename,
+ dir, bitspersample);
+ }
+ break;
+
+
+ /* The reported value is not recognized. */
+ default:
+ error(EXIT_FAILURE, 0, "%s: %s (dir %zu): value %u not recognized "
+ "for SAMPLEFORMAT tag", __func__, filename, dir, sampleformat);
+ }
+
+ /* Control should never reach here, but to avoid compiler warnings (and
+ hard to find bugs when it does reach here), we'll just return an
+ invalid type. */
+ return GAL_TYPE_INVALID;
+}
+
+
+
+
+
+/* Get the basic TIFF image information. */
+static void
+tiff_img_info(TIFF *tif, uint8_t *type, size_t *ndim, size_t *dsize,
+ size_t *numch, char *filename, size_t dir)
+{
+ size_t d=0;
+ uint16_t u16;
+ uint32_t u32;
+
+ /* Based on if `IMAGEDEPTH' is defined in the TIFF header, set the
+ dimensions. */
+ if( TIFFGetField(tif, TIFFTAG_IMAGEDEPTH, &u32) )
+ dsize[ d++ ]=u32;
+
+ /* Read the other sizes. Note that in the TIFF standard, IMAGELENGTH is
+ the vertical length of the image and IMAGEWIDTH is the horizontal
+ length. */
+ tiff_read_tag(tif, TIFFTAG_IMAGELENGTH, &u32, filename, dir);
+ dsize[ d++ ]=u32;
+ tiff_read_tag(tif, TIFFTAG_IMAGEWIDTH, &u32, filename, dir);
+ dsize[ d++ ]=u32;
+
+ /* Write the dimensions. */
+ *ndim=d;
+
+ /* Read the type of the image. */
+ *type=tiff_type_read(tif, filename, dir);
+
+ /* Read the number of channels in the image. */
+ *numch = TIFFGetField(tif, TIFFTAG_SAMPLESPERPIXEL, &u16) ? u16 : 1;
+}
+
+
+
+
+
+/* Based on the `TIFFReadContigStripData' function of `tools/tiffinfo.c' of
+ Libtiff's source. */
+void
+tiff_read_contig_strip_data(TIFF *tif, char *filename, size_t dir,
+ gal_data_t *out, size_t numch)
+{
+ tstrip_t strip;
+ size_t ostart=0;
+ unsigned char *buf;
+ uint32_t row, rowsperstrip = (uint32_t)-1;
+ size_t nrow, scanline=TIFFScanlineSize(tif);
+ uint32 h=out->ndim==2?out->dsize[0]:out->dsize[1];
+
+ /* Allocate the buffer. */
+ errno=0;
+ buf = (unsigned char *)_TIFFmalloc(TIFFStripSize(tif));
+ if(buf==NULL)
+ error(EXIT_FAILURE, errno, "%s: %s (dir %zu): couldn't allocate "
+ "necessary space to load image (%zu bytes)", __func__, filename,
+ dir, scanline);
+
+ /* Parse over the rows and read everything. */
+ TIFFGetField(tif, TIFFTAG_ROWSPERSTRIP, &rowsperstrip);
+ for(row=0; row<h; row+=rowsperstrip)
+ {
+ /* Read the part of the image into the buffer. */
+ strip = TIFFComputeStrip(tif, row, 0);
+ nrow = (row+rowsperstrip > h ? h-row : rowsperstrip);
+ if( TIFFReadEncodedStrip(tif, strip, buf, nrow*scanline) < 0 )
+ error(EXIT_FAILURE, 0, "%s: %s (dir %zu): couldn't read data",
+ __func__, filename, dir);
+
+ /* Copy the contents of the buffer to the output array. Note that
+ `ostart' is the byte count already, so the type is
+ irrelevant. Thus, we can read `out->array' as a `char *'
+ pointer.*/
+ memcpy( (char *)(out->array)+ostart, buf, nrow*scanline);
+ ostart+=nrow*scanline;
+ }
+
+ /* Clean up. */
+ _TIFFfree(buf);
+}
+
+
+
+
+
+/* Based on the `TIFFReadSeparateStripData' function of `tools/tiffinfo.c'
+ of Libtiff's source. */
+static void
+tiff_read_separate_strip_data(TIFF* tif, char *filename, size_t dir,
+ gal_data_t *out)
+{
+ tsample_t s;
+ gal_data_t *ch;
+ tstrip_t strip;
+ unsigned char *buf;
+ uint32_t rowsperstrip = (uint32_t)-1;
+ size_t nrow, scanline = TIFFScanlineSize(tif);
+ size_t ostart=0, numch=gal_list_data_number(out);
+ uint32 row, h=out->ndim==2?out->dsize[0]:out->dsize[1];
+
+ /* Allocate the buffer. */
+ errno=0;
+ buf = (unsigned char *)_TIFFmalloc(TIFFStripSize(tif));
+ if(buf==NULL)
+ error(EXIT_FAILURE, errno, "%s: %s (dir %zu): couldn't allocate "
+ "necessary space to load image (%zu bytes)", __func__, filename,
+ dir, scanline);
+
+ /* Parse over the dataset and read them into the output. */
+ TIFFGetField(tif, TIFFTAG_ROWSPERSTRIP, &rowsperstrip);
+ for(row=0; row<h; row+=rowsperstrip)
+ {
+ /* Restart the channel list. */
+ ch=out;
+
+ /* Go over each channel. */
+ for(s=0; s<numch; s++)
+ {
+ /* Read this part of the data into the buffer. */
+ strip = TIFFComputeStrip(tif, row, s);
+ nrow = (row+rowsperstrip > h ? h-row : rowsperstrip);
+ if( TIFFReadEncodedStrip(tif, strip, buf, nrow*scanline) < 0 )
+ error(EXIT_FAILURE, 0, "%s: %s (dir %zu): couldn't read data",
+ __func__, filename, dir);
+
+ /* Write the buffer into the output array. */
+ memcpy( (char *)(ch->array)+ostart, buf, nrow*scanline);
+
+ /* Go onto the next channel. */
+ ch=ch->next;
+ }
+
+ /* Increment the starting pointer. */
+ ostart+=nrow*scanline;
+ }
+
+ /* Clean up. */
+ _TIFFfree(buf);
+}
+
+
+
+
+
+/* The data have been read contiguously (the pixels for each color are
+ beside each other). We need to separate the color channels into
+ different datasets. We will also use this chance to reverse the order of
+ the rows */
+static gal_data_t *
+tiff_separate_channels_reverse(gal_data_t *out, size_t numch,
+ size_t minmapsize)
+{
+ size_t k, l, i, j;
+ gal_data_t *tch, *ch=NULL;
+ size_t so=gal_type_sizeof(out->type);
+ size_t width=out->dsize[1]*so/numch, lwidth=out->dsize[1]*so;
+
+ /* A small sanity check. */
+ if(out->ndim==3)
+ error(EXIT_FAILURE, 0, "%s: currently only 2D datasets are supported, "
+ "please get in touch with us at %s to add 3D support", __func__,
+ PACKAGE_BUGREPORT);
+
+
+ /* Make the separated datasets (temporarily fix the extra width). */
+ out->dsize[1] /= numch;
+ for(k=0; k<numch; ++k)
+ gal_list_data_add_alloc(&ch, NULL, out->type, out->ndim, out->dsize,
+ NULL, 0, minmapsize, NULL, NULL, NULL);
+ out->dsize[1] *= numch;
+
+
+ /* Parse over the rows and write them in the output. */
+ for(i=0;i<out->dsize[0];++i)
+ {
+ /* `j' is the output row. */
+ j=out->dsize[0]-1-i;
+
+ /* `k' is the element in each row and `l' is the color/channel. */
+ for(k=0;k<ch->dsize[1];++k)
+ {
+ /* Initialize the color/channel counter ocpy the elements. */
+ l=0;
+ for(tch=ch; tch!=NULL; tch=tch->next)
+ {
+ /* Elements of the `j'th row into the `i'th. */
+ memcpy( (char *)(tch->array) + i*width + k*so,
+ (char *)(out->array) + j*lwidth + k*numch*so + so*l,
+ so);
+
+ /* Increment the color. */
+ ++l;
+ }
+ }
+ }
+
+ /* Clean up and return. */
+ return ch;
+}
+
+
+
+
+
+/* The standard TIFF format is up-side-down when viewed in FITS (which is
+ the base of Gnuastro also). So we need to reverse the array for an
+ identical orientation. */
+static void
+tiff_reverse_rows(gal_data_t *out)
+{
+ gal_data_t *ch=out;
+ size_t c, i, j, numch=gal_list_data_number(out);
+ size_t width=out->dsize[1]*gal_type_sizeof(out->type);
+ void *tmp=gal_data_malloc_array(out->type, out->dsize[1], __func__, "tmp");
+
+ /* A small sanity check. */
+ if(out->ndim==3)
+ error(EXIT_FAILURE, 0, "%s: currently only 2D datasets are supported, "
+ "please get in touch with us at %s to add 3D support", __func__,
+ PACKAGE_BUGREPORT);
+
+ /* Parse over the rows and reverse them. */
+ for(c=0;c<numch;++c)
+ {
+ /* Initialize the parsing counters. */
+ i=0;
+ j=out->dsize[0]-1;
+
+ /* Go over the channel. */
+ while(j>i)
+ {
+ /* Copy the `i'th row into a temporary array. */
+ memcpy(tmp, (char *)(ch->array)+i*width, width);
+
+ /* Put the `j'th row into the `i'th row. */
+ memcpy( (char *)(ch->array)+i*width, (char *)(ch->array)+j*width,
+ width );
+
+ /* Put the `tmp' row into `j'. */
+ memcpy( (char *)(ch->array)+j*width, tmp, width);
+
+ /* Increment the points. */
+ ++i;
+ --j;
+ }
+
+ /* Go to the next channel. */
+ ch=ch->next;
+ }
+
+ /* Clean up. */
+ free(tmp);
+}
+
+
+
+
+
+/* Read the data following the `TIFFReadData' of Libtiff's
+ `tools/tiffinfo.c' in the libtiff source code. */
+static gal_data_t *
+tiff_img_read(TIFF *tif, char *filename, size_t dir, size_t minmapsize)
+{
+ uint8_t type;
+ uint16_t config;
+ gal_data_t *sep, *out=NULL;
+ size_t i, ndim, numch, dsize[3];
+
+
+ /* Get the basic image information. */
+ tiff_img_info(tif, &type, &ndim, dsize, &numch, filename, dir);
+
+
+ /* Find the planar state of the input: are the channels separate or
+ contiguous? Based on that, allocate the output data structure. */
+ TIFFGetField(tif, TIFFTAG_PLANARCONFIG, &config);
+ if( config==PLANARCONFIG_CONTIG )
+ {
+ /* We'll multiply the last dimension's length by the number of
+ channels to keep the contiguous color information in one array. */
+ dsize[ndim-1] *= numch;
+ out=gal_data_alloc(NULL, type, ndim, dsize, NULL, 0, minmapsize,
+ NULL, NULL, NULL);
+ }
+ else
+ for(i=0; i<numch; ++i)
+ gal_list_data_add_alloc(&out, NULL, type, ndim, dsize, NULL, 0,
+ minmapsize, NULL, NULL, NULL);
+
+
+ /* The reading of the dataset depends on how it is organized, so first
+ we'll look into the `planarconfig' field. */
+ if( TIFFIsTiled(tif) )
+ {
+ if(config==PLANARCONFIG_CONTIG)
+ error(EXIT_FAILURE, 0, "%s: %s (dir %zu) is a contiguous tiled TIFF "
+ "file which is not yet supported in Gnuastro, please get in "
+ "touch with us at %s to add this feature", __func__, filename,
+ dir, PACKAGE_BUGREPORT);
+ else
+ error(EXIT_FAILURE, 0, "%s: %s (dir %zu) is a non-contiguous tiled "
+ "TIFF file which is not yet supported in Gnuastro, please "
+ "get in touch with us at %s to add this feature", __func__,
+ filename, dir, PACKAGE_BUGREPORT);
+ }
+ else
+ {
+ if(config==PLANARCONFIG_CONTIG)
+ tiff_read_contig_strip_data(tif, filename, dir, out, numch);
+ else
+ tiff_read_separate_strip_data(tif, filename, dir, out);
+ }
+
+
+ /* When there are more than one channels and the colors are stored
+ contiguously, we need to break up the array into multiple arrays. When
+ any of these conditions don't hold, the channels are already
+ separated, we just need to reverse them.*/
+ if( numch>1 && config==PLANARCONFIG_CONTIG )
+ {
+ sep=tiff_separate_channels_reverse(out, numch, minmapsize);
+ gal_data_free(out);
+ out=sep;
+ }
+ else
+ tiff_reverse_rows(out);
+
+
+ /* Return the output. */
+ return out;
+}
+#endif
+
+
+
+
+
+gal_data_t *
+gal_tiff_read(char *filename, size_t dir, size_t minmapsize)
+{
+#ifdef HAVE_LIBTIFF
+ TIFF *tif;
+ gal_data_t *out;
+ size_t dircount=0;
+
+ /* Open the TIFF file. */
+ tif=TIFFOpen(filename, "r");
+ if(tif==NULL)
+ error(EXIT_FAILURE, 0, "%s: `%s' couldn't be opened for reading",
+ __func__, filename);
+
+ /* If anything other than the first directory (value of zero) is
+ requested, then change the directories. */
+ if(dir)
+ {
+ if( TIFFSetDirectory(tif, dir)==0 )
+ {
+ /* For an informative error message, count how many directories
+ are in the TIFF file. */
+ do ++dircount; while( TIFFReadDirectory(tif) );
+
+ /* Close the TIFF file and return. */
+ TIFFClose(tif);
+ error(EXIT_FAILURE, 0, "%s: `%s' has %zu director%s/extension%s, "
+ "and directories are counted from 0. You have asked for "
+ "directory %zu", __func__, filename, dircount,
+ dircount==1?"y":"ies", dircount==1?"":"s", dir);
+ }
+ }
+
+ /* Read the image. */
+ out=tiff_img_read(tif, filename, dir, minmapsize);
+
+ /* Close file, clean up and return. */
+ TIFFClose(tif);
+ return out;
+#else
+ error(EXIT_FAILURE, 0, "%s: libtiff was not found during the "
+ "configuration of %s on this system. To read from TIFF files, "
+ "libtiff is required. Please install libtiff, then configure, make "
+ "and install %s again", __func__, PACKAGE_STRING, PACKAGE_STRING);
+ return NULL;
+#endif /* HAVE_LIBTIFF */
+}
diff --git a/lib/wcs.c b/lib/wcs.c
index ededd0d..070fb9a 100644
--- a/lib/wcs.c
+++ b/lib/wcs.c
@@ -183,6 +183,10 @@ gal_wcs_read(char *filename, char *hdu, size_t hstartwcs,
fitsfile *fptr;
struct wcsprm *wcs;
+ /* Make sure we are dealing with a FITS file. */
+ if( gal_fits_name_is_fits(filename) == 0 )
+ return NULL;
+
/* Check HDU for realistic conditions: */
fptr=gal_fits_hdu_open_format(filename, hdu, 0);
diff --git a/tests/Makefile.am b/tests/Makefile.am
index 6e17151..dc00102 100644
--- a/tests/Makefile.am
+++ b/tests/Makefile.am
@@ -53,12 +53,13 @@ if COND_HASGNULIBTOOL
endif
if COND_ARITHMETIC
MAYBE_ARITHMETIC_TESTS = arithmetic/snimage.sh arithmetic/onlynumbers.sh \
- arithmetic/where.sh arithmetic/or.sh
+ arithmetic/where.sh arithmetic/or.sh arithmetic/connected-components.sh
arithmetic/onlynumbers.sh: prepconf.sh.log
+ arithmetic/connected-components.sh: noisechisel/noisechisel.sh.log
arithmetic/snimage.sh: noisechisel/noisechisel.sh.log
arithmetic/where.sh: noisechisel/noisechisel.sh.log
- arithmetic/or.sh: noisechisel/noisechisel.sh.log
+ arithmetic/or.sh: segment/segment.sh.log
endif
if COND_BUILDPROG
MAYBE_BUILDPROG_TESTS = buildprog/simpleio.sh
@@ -124,11 +125,12 @@ if COND_MATCH
match/merged-cols.sh: prepconf.sh.log
endif
if COND_MKCATALOG
- MAYBE_MKCATALOG_TESTS = mkcatalog/simple.sh mkcatalog/simple-3d.sh \
- mkcatalog/aperturephot.sh
+ MAYBE_MKCATALOG_TESTS = mkcatalog/detections.sh mkcatalog/simple-3d.sh \
+ mkcatalog/objects-clumps.sh mkcatalog/aperturephot.sh
- mkcatalog/simple.sh: noisechisel/noisechisel.sh.log
- mkcatalog/simple-3d.sh: noisechisel/noisechisel-3d.sh.log
+ mkcatalog/objects-clumps.sh: segment/segment.sh.log
+ mkcatalog/detections.sh: arithmetic/connected-components.sh.log
+ mkcatalog/simple-3d.sh: segment/segment-3d.sh.log
mkcatalog/aperturephot.sh: noisechisel/noisechisel.sh.log \
mkprof/clearcanvas.sh.log
endif
@@ -161,6 +163,12 @@ if COND_NOISECHISEL
noisechisel/noisechisel.sh: mknoise/addnoise.sh.log
noisechisel/noisechisel-3d.sh: mknoise/addnoise-3d.sh.log
endif
+if COND_SEGMENT
+ MAYBE_SEGMENT_TESTS = segment/segment.sh segment/segment-3d.sh
+
+ segment/segment.sh: noisechisel/noisechisel.sh.log
+ segment/segment-3d.sh: noisechisel/noisechisel-3d.sh.log
+endif
if COND_STATISTICS
MAYBE_STATISTICS_TESTS = statistics/basicstats.sh statistics/estimate_sky.sh
@@ -228,13 +236,14 @@ lib/multithread.sh: mkprof/mosaic1.sh.log
# Final Tests
# ===========
-TESTS = prepconf.sh lib/multithread.sh lib/versioncxx.sh \
+TESTS = prepconf.sh lib/multithread.sh lib/versioncxx.sh \
$(MAYBE_ARITHMETIC_TESTS) $(MAYBE_BUILDPROG_TESTS) \
$(MAYBE_CONVERTT_TESTS) $(MAYBE_CONVOLVE_TESTS) $(MAYBE_COSMICCAL_TESTS) \
$(MAYBE_CROP_TESTS) $(MAYBE_FITS_TESTS) $(MAYBE_MATCH_TESTS) \
$(MAYBE_MKCATALOG_TESTS) $(MAYBE_MKNOISE_TESTS) $(MAYBE_MKPROF_TESTS) \
- $(MAYBE_NOISECHISEL_TESTS) $(MAYBE_STATISTICS_TESTS) \
- $(MAYBE_SUBTRACTSKY_TESTS) $(MAYBE_TABLE_TESTS) $(MAYBE_WARP_TESTS)
+ $(MAYBE_NOISECHISEL_TESTS) $(MAYBE_SEGMENT_TESTS) \
+ $(MAYBE_STATISTICS_TESTS) $(MAYBE_SUBTRACTSKY_TESTS) \
+ $(MAYBE_TABLE_TESTS) $(MAYBE_WARP_TESTS)
diff --git a/tests/arithmetic/where.sh
b/tests/arithmetic/connected-components.sh
similarity index 85%
copy from tests/arithmetic/where.sh
copy to tests/arithmetic/connected-components.sh
index cb5acc1..dfdf058 100755
--- a/tests/arithmetic/where.sh
+++ b/tests/arithmetic/connected-components.sh
@@ -1,4 +1,4 @@
-# Mask non-detected pixels in the image with the `where' operator.
+# Find the connected components in NoiseChisel's output.
#
# See the Tests subsection of the manual for a complete explanation
# (in the Installing gnuastro section).
@@ -24,7 +24,7 @@
# file exists (basicchecks.sh is in the source tree).
prog=arithmetic
execname=../bin/$prog/ast$prog
-img=convolve_spatial_noised_labeled.fits
+img=convolve_spatial_noised_detected.fits
@@ -49,4 +49,5 @@ if [ ! -f $img ]; then echo "$img does not exist.";
exit 77; fi
# Actual test script
# ==================
-$execname $img $img 0 eq nan where -h1 -h2 --output=where.fits
+$execname $img 2 connected-components -hDETECTIONS \
+ --output=connected-components.fits
diff --git a/tests/arithmetic/or.sh b/tests/arithmetic/or.sh
index 81efa6f..9437258 100755
--- a/tests/arithmetic/or.sh
+++ b/tests/arithmetic/or.sh
@@ -24,7 +24,7 @@
# file exists (basicchecks.sh is in the source tree).
prog=arithmetic
execname=../bin/$prog/ast$prog
-img=convolve_spatial_noised_labeled.fits
+img=convolve_spatial_noised_detected_segmented.fits
@@ -49,4 +49,4 @@ if [ ! -f $img ]; then echo "$img does not exist.";
exit 77; fi
# Actual test script
# ==================
-$execname $img 6 eq $img 3 eq or -h2 -h2 --output=or.fits
+$execname $img 1 eq $img 3 eq or -gOBJECTS --output=or.fits
diff --git a/tests/arithmetic/snimage.sh b/tests/arithmetic/snimage.sh
index 18f719a..37dcb2d 100755
--- a/tests/arithmetic/snimage.sh
+++ b/tests/arithmetic/snimage.sh
@@ -24,7 +24,8 @@
# file exists (basicchecks.sh is in the source tree).
prog=arithmetic
execname=../bin/$prog/ast$prog
-img=convolve_spatial_noised_labeled.fits
+imgin=convolve_spatial_noised.fits
+imgnc=convolve_spatial_noised_detected.fits
@@ -41,7 +42,8 @@ img=convolve_spatial_noised_labeled.fits
# - The input data was not made (for example the test that created the
# data file failed).
if [ ! -f $execname ]; then echo "$execname not created."; exit 77; fi
-if [ ! -f $img ]; then echo "$img does not exist."; exit 77; fi
+if [ ! -f $imgin ]; then echo "$imgin does not exist."; exit 77; fi
+if [ ! -f $imgnc ]; then echo "$imgnc does not exist."; exit 77; fi
@@ -49,5 +51,5 @@ if [ ! -f $img ]; then echo "$img does not exist.";
exit 77; fi
# Actual test script
# ==================
-$execname $img $img - $img / --hdu=1 --hdu=4 --hdu=5 \
+$execname $imgin $imgnc - $imgnc / --hdu=1 --hdu=SKY --hdu=SKY_STD \
--output=snimage.fits
diff --git a/tests/arithmetic/where.sh b/tests/arithmetic/where.sh
index cb5acc1..a8948b0 100755
--- a/tests/arithmetic/where.sh
+++ b/tests/arithmetic/where.sh
@@ -24,7 +24,7 @@
# file exists (basicchecks.sh is in the source tree).
prog=arithmetic
execname=../bin/$prog/ast$prog
-img=convolve_spatial_noised_labeled.fits
+img=convolve_spatial_noised_detected.fits
diff --git a/tests/buildprog/simpleio.c b/tests/buildprog/simpleio.c
index 7354ba7..11f6597 100644
--- a/tests/buildprog/simpleio.c
+++ b/tests/buildprog/simpleio.c
@@ -39,7 +39,7 @@ main(int argc, char *argv[])
}
/* Read the image into memory. */
- image=gal_fits_img_read(argv[1], argv[2], -1, 0, 0);
+ image=gal_fits_img_read(argv[1], argv[2], -1);
/* Let the user know. */
printf("%s (hdu %s) is read into memory.\n", argv[1], argv[2]);
diff --git a/tests/lib/multithread.c b/tests/lib/multithread.c
index 65cd5a6..42f632c 100644
--- a/tests/lib/multithread.c
+++ b/tests/lib/multithread.c
@@ -95,7 +95,7 @@ main(void)
/* Read the image into memory as a float32 data type. */
- p.image=gal_fits_img_read_to_type(filename, hdu, GAL_TYPE_FLOAT32, -1,0,0);
+ p.image=gal_fits_img_read_to_type(filename, hdu, GAL_TYPE_FLOAT32, -1);
/* Print some basic information before the actual contents: */
diff --git a/tests/mkcatalog/aperturephot.sh b/tests/mkcatalog/aperturephot.sh
index 196049b..c331029 100755
--- a/tests/mkcatalog/aperturephot.sh
+++ b/tests/mkcatalog/aperturephot.sh
@@ -25,7 +25,7 @@
prog=mkcatalog
objimg=clearcanvas.fits
execname=../bin/$prog/ast$prog
-img=convolve_spatial_noised_labeled.fits
+img=convolve_spatial_noised_detected.fits
@@ -51,6 +51,6 @@ if [ ! -f $objimg ]; then echo "$objimg does not exist";
exit 77; fi
# Actual test script
# ==================
-$execname $img --objectsfile=$objimg --objectshdu=1 \
- --output=aperturephot.fits \
+$execname $objimg --hdu=1 --valuesfile=$img \
+ --output=aperturephot.fits \
--objid --x --y --ra --dec --magnitude --sn
diff --git a/tests/mkcatalog/aperturephot.sh b/tests/mkcatalog/detections.sh
similarity index 77%
copy from tests/mkcatalog/aperturephot.sh
copy to tests/mkcatalog/detections.sh
index 196049b..5077dee 100755
--- a/tests/mkcatalog/aperturephot.sh
+++ b/tests/mkcatalog/detections.sh
@@ -23,9 +23,9 @@
# basic checks to see if the executable is made or if the defaults
# file exists (basicchecks.sh is in the source tree).
prog=mkcatalog
-objimg=clearcanvas.fits
execname=../bin/$prog/ast$prog
-img=convolve_spatial_noised_labeled.fits
+labels=connected-components.fits
+base=convolve_spatial_noised_detected.fits
@@ -42,8 +42,8 @@ img=convolve_spatial_noised_labeled.fits
# - The input data was not made (for example the test that created the
# data file failed).
if [ ! -f $execname ]; then echo "$execname not created."; exit 77; fi
-if [ ! -f $img ]; then echo "$img does not exist."; exit 77; fi
-if [ ! -f $objimg ]; then echo "$objimg does not exist"; exit 77; fi
+if [ ! -f $labels ]; then echo "$labels does not exist."; exit 77; fi
+if [ ! -f $base ]; then echo "$base does not exist."; exit 77; fi
@@ -51,6 +51,5 @@ if [ ! -f $objimg ]; then echo "$objimg does not exist";
exit 77; fi
# Actual test script
# ==================
-$execname $img --objectsfile=$objimg --objectshdu=1 \
- --output=aperturephot.fits \
- --objid --x --y --ra --dec --magnitude --sn
+$execname $labels -h1 --valuesfile=$base --tableformat=txt \
+ --output=detections.txt --x --y --ra --dec --magnitude --sn
diff --git a/tests/mkcatalog/simple.sh b/tests/mkcatalog/objects-clumps.sh
similarity index 91%
rename from tests/mkcatalog/simple.sh
rename to tests/mkcatalog/objects-clumps.sh
index 71a3ebb..987d5e7 100755
--- a/tests/mkcatalog/simple.sh
+++ b/tests/mkcatalog/objects-clumps.sh
@@ -24,7 +24,7 @@
# file exists (basicchecks.sh is in the source tree).
prog=mkcatalog
execname=../bin/$prog/ast$prog
-img=convolve_spatial_noised_labeled.fits
+img=convolve_spatial_noised_detected_segmented.fits
@@ -50,4 +50,4 @@ if [ ! -f $img ]; then echo "$img does not exist.";
exit 77; fi
# Actual test script
# ==================
$execname $img --x --y --ra --dec --magnitude --upperlimitmag --sn \
- --tableformat=txt
+ --tableformat=txt --clumpscat --output=objects-clumps.txt
diff --git a/tests/mkcatalog/simple-3d.sh b/tests/mkcatalog/simple-3d.sh
index e89b205..75e2986 100755
--- a/tests/mkcatalog/simple-3d.sh
+++ b/tests/mkcatalog/simple-3d.sh
@@ -24,7 +24,7 @@
# file exists (basicchecks.sh is in the source tree).
prog=mkcatalog
execname=../bin/$prog/ast$prog
-img=3d-cat_noised_labeled.fits
+img=3d-cat_noised_detected_segmented.fits
@@ -50,4 +50,4 @@ if [ ! -f $img ]; then echo "$img does not exist.";
exit 77; fi
# Actual test script
# ==================
$execname $img --x --y --z --w1 --w2 --w3 --area --brightness --sn \
- --upperlimit --tableformat=txt
+ --upperlimit
diff --git a/tests/noisechisel/noisechisel.sh b/tests/noisechisel/noisechisel.sh
index ef9ebdf..d2bf1b2 100755
--- a/tests/noisechisel/noisechisel.sh
+++ b/tests/noisechisel/noisechisel.sh
@@ -1,4 +1,4 @@
-# Detect objects and clumps in an image using NoiseChisel.
+# Detect objects in an image using NoiseChisel.
#
# See the Tests subsection of the manual for a complete explanation
# (in the Installing gnuastro section).
@@ -41,7 +41,7 @@ img=convolve_spatial_noised.fits
# - The input data was not made (for example the test that created the
# data file failed).
if [ ! -f $execname ]; then echo "$execname not created."; exit 77; fi
-if [ ! -f $img ]; then echo "$img does not exist."; exit 77; fi
+if [ ! -f $img ]; then echo "$img does not exist."; exit 77; fi
@@ -49,5 +49,5 @@ if [ ! -f $img ]; then echo "$img does not exist.";
exit 77; fi
# Actual test script
# ==================
-$execname $img --cleangrowndet --checkdetection --checksegmentation \
- --continueaftercheck --tilesize=100,100
+$execname $img --tilesize=100,100 --snquant=0.999 --cleangrowndet \
+ --checkdetection --continueaftercheck
diff --git a/tests/prepconf.sh b/tests/prepconf.sh
index 0035607..d82d8de 100755
--- a/tests/prepconf.sh
+++ b/tests/prepconf.sh
@@ -72,8 +72,9 @@ rm addedoptions.txt
# easy readability. Note that some programs may need to build their
# configuration files during compilation. Hence, their configuration files
# are in the build directory, not the source directory.
-for prog in arithmetic buildprog convertt convolve cosmiccal crop fits \
- match mkcatalog mknoise mkprof noisechisel statistics table warp
+for prog in arithmetic buildprog convertt convolve cosmiccal crop fits \
+ match mkcatalog mknoise mkprof noisechisel segment \
+ statistics table warp
do
if test -f $topsrc/bin/$prog/ast$prog.conf; then
ctopdir=$topsrc
diff --git a/tests/arithmetic/where.sh b/tests/segment/segment-3d.sh
similarity index 85%
copy from tests/arithmetic/where.sh
copy to tests/segment/segment-3d.sh
index cb5acc1..871f108 100755
--- a/tests/arithmetic/where.sh
+++ b/tests/segment/segment-3d.sh
@@ -1,4 +1,4 @@
-# Mask non-detected pixels in the image with the `where' operator.
+# Segment the 3D detections into clumps and objects.
#
# See the Tests subsection of the manual for a complete explanation
# (in the Installing gnuastro section).
@@ -22,9 +22,9 @@
# Set the variables (The executable is in the build tree). Do the
# basic checks to see if the executable is made or if the defaults
# file exists (basicchecks.sh is in the source tree).
-prog=arithmetic
+prog=segment
execname=../bin/$prog/ast$prog
-img=convolve_spatial_noised_labeled.fits
+img=3d-cat_noised_detected.fits
@@ -49,4 +49,5 @@ if [ ! -f $img ]; then echo "$img does not exist.";
exit 77; fi
# Actual test script
# ==================
-$execname $img $img 0 eq nan where -h1 -h2 --output=where.fits
+$execname $img --largetilesize=100,100,30 --snquant=0.99 \
+ --config=.gnuastro/astsegment-3d.conf
diff --git a/tests/arithmetic/or.sh b/tests/segment/segment.sh
similarity index 87%
copy from tests/arithmetic/or.sh
copy to tests/segment/segment.sh
index 81efa6f..995dfa3 100755
--- a/tests/arithmetic/or.sh
+++ b/tests/segment/segment.sh
@@ -1,4 +1,4 @@
-# Choose two detected regions with the `or' operator
+# Segment the detections into clumps and objects.
#
# See the Tests subsection of the manual for a complete explanation
# (in the Installing gnuastro section).
@@ -22,9 +22,9 @@
# Set the variables (The executable is in the build tree). Do the
# basic checks to see if the executable is made or if the defaults
# file exists (basicchecks.sh is in the source tree).
-prog=arithmetic
+prog=segment
execname=../bin/$prog/ast$prog
-img=convolve_spatial_noised_labeled.fits
+img=convolve_spatial_noised_detected.fits
@@ -49,4 +49,4 @@ if [ ! -f $img ]; then echo "$img does not exist.";
exit 77; fi
# Actual test script
# ==================
-$execname $img 6 eq $img 3 eq or -h2 -h2 --output=or.fits
+$execname $img --tilesize=100,100 --snquant=0.99
- [gnuastro-commits] master 175354f 095/113: All MakeCatalog spectrums set to NaN when no measurement was done, (continued)
- [gnuastro-commits] master 175354f 095/113: All MakeCatalog spectrums set to NaN when no measurement was done, Mohammad Akhlaghi, 2021/04/16
- [gnuastro-commits] master fe0e594 097/113: Imported recent work in master, no conflicts, Mohammad Akhlaghi, 2021/04/16
- [gnuastro-commits] master 5d5fd95 106/113: NoiseChisel: independent test of input's dimensions, Mohammad Akhlaghi, 2021/04/16
- [gnuastro-commits] master ee4bf1e 069/113: Imported recent work from master, Segment's 3D kernels added, Mohammad Akhlaghi, 2021/04/16
- [gnuastro-commits] master 2c6c7e0 076/113: Imported recent work, no conflicts, Mohammad Akhlaghi, 2021/04/16
- [gnuastro-commits] master e2aba57 083/113: Recent work in master imported, conflicts fixed, Mohammad Akhlaghi, 2021/04/16
- [gnuastro-commits] master e92fd9c 039/113: Merged recent work in master, conflicts fixed, Mohammad Akhlaghi, 2021/04/16
- [gnuastro-commits] master 54b099d 043/113: Neighbors options doc generalized for different dimensions, Mohammad Akhlaghi, 2021/04/16
- [gnuastro-commits] master 5cd3c1e 044/113: Recent work in master merged, small conflicts fixed, Mohammad Akhlaghi, 2021/04/16
- [gnuastro-commits] master 1b0046b 051/113: Recent additions in master imported here, no conflicts, Mohammad Akhlaghi, 2021/04/16
- [gnuastro-commits] master b1ea932 054/113: Separate Segment program now available in 3D,
Mohammad Akhlaghi <=
- [gnuastro-commits] master f537368 066/113: Segment's default 3D configuration file in tarball, Mohammad Akhlaghi, 2021/04/16
- [gnuastro-commits] master 133e277 071/113: Imported recent work from master, conflicts fixed, Mohammad Akhlaghi, 2021/04/16
- [gnuastro-commits] master a42ac37 085/113: Fixed MakeProfiles loop in checking radius values, Mohammad Akhlaghi, 2021/04/16
- [gnuastro-commits] master 1f9d941 090/113: Imported recent work in master, conflicts fixed, Mohammad Akhlaghi, 2021/04/16
- [gnuastro-commits] master 737a0cd 046/113: Merged recent corrections in master branch, Mohammad Akhlaghi, 2021/04/16
- [gnuastro-commits] master 486f915 067/113: Imported recent work in master, conflicts fixed, Mohammad Akhlaghi, 2021/04/16
- [gnuastro-commits] master d6051a1 074/113: Upperlimit magnitude in 3D, corrected creation of check table, Mohammad Akhlaghi, 2021/04/16
- [gnuastro-commits] master 5dac3c3 075/113: Recent work in master imported, small conflict in book fixed, Mohammad Akhlaghi, 2021/04/16
- [gnuastro-commits] master 3882956 077/113: Narrow-band areas from MakeCatalog in 3D datasets, Mohammad Akhlaghi, 2021/04/16
- [gnuastro-commits] master abe82cb 086/113: Imported recent work in master, no conflicts, Mohammad Akhlaghi, 2021/04/16