[Date Prev][Date Next][Thread Prev][Thread Next][Date Index][Thread Index]
[gnuastro-commits] master 5a2f382: New Segment, NoiseChisel only detecti
|
From: |
Mohammad Akhlaghi |
|
Subject: |
[gnuastro-commits] master 5a2f382: New Segment, NoiseChisel only detection, MakeCatalog re-write |
|
Date: |
Thu, 5 Apr 2018 20:00:24 -0400 (EDT) |
branch: master
commit 5a2f382f4250262c983faf35c69c396eb6513d2c
Author: Mohammad Akhlaghi <address@hidden>
Commit: Mohammad Akhlaghi <address@hidden>
New Segment, NoiseChisel only detection, MakeCatalog re-write
Until now, NoiseChisel was in charge of both detection and
segmentation. This was making it extremely large and hard to manage, both
for the users (in terms of effectively understanding/using it) and the
developers (making it hard to add new features/options). Therefore
NoiseChisel's segmentation part has been spinned off into a new program
called Segment. It already has some good features (like giving an S/N of
true clumps to avoid having to find the S/N from the Sky regions)
As a result of this change in NoiseChisel and the new Segment program,
MakeCatalog (which uses the inputs of these programs) also underwent some
major changes in line with some several old tasks that were defined
before. In particular, it is now much more memory and CPU efficient: it
will only read the datasets that the output columns need and (when
necessary), it can read the Sky and Sky Standard deviation images from a
tiled image (with one pixel per tile).
This has been a very major commit and will hopefully open the way for many
more exciting features in NoiseChisel, Segment and MakeCatalog.
The General Program Usage Tutorial has not yet been corrected to
incorporate the new changes. It will be corrected shortly.
This completes tasks #13759, #14164, and #14869.
---
Makefile.am | 15 +-
NEWS | 43 +
README | 11 +-
bin/TEMPLATE/Makefile.am | 4 +-
bin/TEMPLATE/TEMPLATE.c | 4 +-
bin/TEMPLATE/TEMPLATE.h | 4 +-
bin/TEMPLATE/args.h | 4 +-
bin/TEMPLATE/authors-cite.h | 4 +-
bin/TEMPLATE/main.c | 8 +-
bin/TEMPLATE/main.h | 14 +-
bin/TEMPLATE/ui.c | 9 +-
bin/TEMPLATE/ui.h | 4 +-
bin/mkcatalog/Makefile.am | 4 +-
bin/mkcatalog/args.h | 104 +-
bin/mkcatalog/astmkcatalog.conf | 7 +-
bin/mkcatalog/columns.c | 522 ++---
bin/mkcatalog/main.c | 2 +-
bin/mkcatalog/main.h | 36 +-
bin/mkcatalog/mkcatalog.c | 774 ++------
bin/mkcatalog/mkcatalog.h | 15 +-
bin/mkcatalog/parse.c | 641 ++++++
bin/mkcatalog/{upperlimit.h => parse.h} | 17 +-
bin/mkcatalog/ui.c | 1117 +++++++----
bin/mkcatalog/ui.h | 17 +-
bin/mkcatalog/upperlimit.c | 319 ++-
bin/mkcatalog/upperlimit.h | 4 +
bin/noisechisel/Makefile.am | 8 +-
bin/noisechisel/args.h | 236 +--
bin/noisechisel/astnoisechisel.conf | 16 +-
bin/noisechisel/authors-cite.h | 2 +-
bin/noisechisel/detection.c | 30 +-
bin/noisechisel/detection.h | 2 +-
bin/noisechisel/main.c | 2 +-
bin/noisechisel/main.h | 42 +-
bin/noisechisel/noisechisel.c | 151 +-
bin/noisechisel/noisechisel.h | 2 +-
bin/noisechisel/sky.c | 31 +-
bin/noisechisel/sky.h | 2 +-
bin/noisechisel/threshold.c | 2 +-
bin/noisechisel/threshold.h | 2 +-
bin/noisechisel/ui.c | 114 +-
bin/noisechisel/ui.h | 28 +-
bin/{TEMPLATE => segment}/Makefile.am | 10 +-
bin/{noisechisel => segment}/args.h | 493 ++---
bin/segment/astsegment.conf | 41 +
bin/{noisechisel => segment}/authors-cite.h | 6 +-
bin/{noisechisel => segment}/clumps.c | 653 ++-----
bin/{noisechisel => segment}/clumps.h | 19 +-
bin/{noisechisel => segment}/main.c | 16 +-
bin/segment/main.h | 104 +
.../segmentation.c => segment/segment.c} | 374 +++-
.../segmentation.h => segment/segment.h} | 10 +-
bin/{noisechisel => segment}/ui.c | 529 ++---
bin/{noisechisel => segment}/ui.h | 67 +-
configure.ac | 100 +-
doc/Makefile.am | 18 +-
doc/gnuastro.texi | 2049 +++++++++++++-------
doc/release-checklist.txt | 19 +
lib/Makefile.am | 31 +-
lib/data.c | 6 +-
lib/gnuastro-internal/commonopts.h | 2 +-
lib/gnuastro-internal/kernel-2d.h | 129 ++
lib/gnuastro/binary.h | 2 +-
lib/gnuastro/dimension.h | 1 +
lib/gnuastro/label.h | 74 +
lib/label.c | 529 +++++
tests/Makefile.am | 18 +-
tests/arithmetic/snimage.sh | 8 +-
tests/mkcatalog/aperturephot.sh | 4 +-
tests/mkcatalog/{simple.sh => detections.sh} | 2 +-
tests/mkcatalog/{simple.sh => objects-clumps.sh} | 4 +-
tests/noisechisel/noisechisel.sh | 8 +-
tests/prepconf.sh | 5 +-
.../{arithmetic/snimage.sh => segment/segment.sh} | 7 +-
74 files changed, 5722 insertions(+), 3989 deletions(-)
diff --git a/Makefile.am b/Makefile.am
index f62192f..d316836 100644
--- a/Makefile.am
+++ b/Makefile.am
@@ -83,6 +83,9 @@ endif
if COND_NOISECHISEL
MAYBE_NOISECHISEL = bin/noisechisel
endif
+if COND_SEGMENT
+ MAYBE_SEGMENT = bin/segment
+endif
if COND_STATISTICS
MAYBE_STATISTICS = bin/statistics
endif
@@ -110,12 +113,12 @@ endif
## conditions above). When `MAYBE_TEMPLATE' is not defined, then Make will
## see it as a blank string and igonore it, so there is no problem with
## having an uncommented `MAYBE_TEMPLATE' as a value in `SUBDIRS'.
-SUBDIRS = bootstrapped/lib $(MAYBE_GNULIBCHECK) lib $(MAYBE_ARITHMETIC) \
- $(MAYBE_BUILDPROG) $(MAYBE_CONVERTT) $(MAYBE_CONVOLVE) \
- $(MAYBE_COSMICCAL) $(MAYBE_CROP) $(MAYBE_FITS) $(MAYBE_MATCH) \
- $(MAYBE_MKCATALOG) $(MAYBE_MKNOISE) $(MAYBE_MKPROF) \
- $(MAYBE_NOISECHISEL) $(MAYBE_STATISTICS) $(MAYBE_TABLE) \
- $(MAYBE_TEMPLATE) $(MAYBE_WARP) doc tests
+SUBDIRS = bootstrapped/lib $(MAYBE_GNULIBCHECK) lib $(MAYBE_ARITHMETIC) \
+ $(MAYBE_BUILDPROG) $(MAYBE_CONVERTT) $(MAYBE_CONVOLVE) \
+ $(MAYBE_COSMICCAL) $(MAYBE_CROP) $(MAYBE_FITS) $(MAYBE_MATCH) \
+ $(MAYBE_MKCATALOG) $(MAYBE_MKNOISE) $(MAYBE_MKPROF) $(MAYBE_NOISECHISEL) \
+ $(MAYBE_SEGMENT) $(MAYBE_STATISTICS) $(MAYBE_TABLE) $(MAYBE_TEMPLATE) \
+ $(MAYBE_WARP) doc tests
diff --git a/NEWS b/NEWS
index 7ecd28a..a426177 100644
--- a/NEWS
+++ b/NEWS
@@ -5,6 +5,9 @@ GNU Astronomy Utilities NEWS -*-
outline -*-
** New features
+ New program:
+ - Segment: new program in charge of segmentation over detections.
+
Arithmetic:
- filter-sigclip-mean: sigma-clipped, mean filter operator.
- filter-sigclip-median: sigma-clipped, median filter operator.
@@ -13,7 +16,20 @@ GNU Astronomy Utilities NEWS -*-
outline -*-
- TIFF images can also be used as input.
MakeCatalog:
+ MakeCatalog will only read the datasets necessary for the requested
+ columns. Until now, it would read all the possible datasets and all the
+ intermediate measurements. This is thus major improvement in memory and
+ CPU usage. As a result, the input argument is no longer assumed to be
+ the values file, but the object labels file. Please see the
+ "MakeCatalog inputs and basic settings" section of the book for
+ more. Here is the summary of the the options:
+ --valuesfile: filename containing the values dataset.
+ --valueshdu: HDU/extension containing the values dataset.
+ --clumpscat: Make a clumps catalog also (`WCLUMPS' keyword not used
anymore).
+ --subtractsky: Subtract the given Sky from the values dataset.
--variance: input standard deviation image is actually variance.
+ --checkupperlimit: make a table for random positions and measurements.
+ --geoarea: the number of labeled pixels (irrespective of value).
--brightnesserr: error in estimating the brightness.
--mean: calculate the mean pixel value within an object or clump.
--median: calculate the median pixel value within an object or clump.
@@ -22,6 +38,9 @@ GNU Astronomy Utilities NEWS -*-
outline -*-
--upperlimitonesigma: 1sigma value of the random distribution.
--upperlimitskew: (mean-median)/sigma or skewness of random distribution.
+ NoiseChisel:
+ --rawoutput: only output the detection labels and Sky and its STD.
+
Statistics:
--manualbinrange: histogram or CFP range can be outside of distribution.
@@ -37,6 +56,8 @@ GNU Astronomy Utilities NEWS -*-
outline -*-
gal_jpeg_suffix_is_jpeg: Returns 1 if given suffix is JPEG.
gal_jpeg_read: Reads input JPEG image into `gal_data_t'.
gal_jpeg_write: Writes a `gal_data_t' into a JPEG file.
+ gal_label_grow_indexs: grow known indexs into desired areas.
+ gal_label_oversegment: apply over-segmentation to an input dataset.
gal_pdf_name_is_pdf: Returns 1 if given filename is PDF.
gal_pdf_suffix_is_pdf: Returns 1 if given suffix is PDF.
gal_pdf_write: Writes a dataset into an PDF file.
@@ -47,6 +68,18 @@ GNU Astronomy Utilities NEWS -*-
outline -*-
** Removed features
+ NoiseChisel:
+
+ - Segmentation (and thus the options below) moved to the new Segment
+ program: --onlydetection, --segsnminarea, --checkclumpsn, --segquant,
+ --keepmaxnearriver, --gthresh, --minriverlength, --objbordersn,
+ --grownclumps, --checksegmentation.
+
+ --skysubtracted: no longer necessary, included in noise measuremnts.
+
+ MakeCatalog:
+ --skysubtracted: no longer necessary, included in noise measuremnts.
+
** Changed features
Fits:
@@ -54,11 +87,21 @@ GNU Astronomy Utilities NEWS -*-
outline -*-
--comment: can be called/written multiple times in one run.
NoiseChisel:
+ Many of the changes below are because NoiseChisel doesn't do
+ segmentation any more.
--kernel: value `none' will disable convolution.
+ - Renamed options:
+ --convolvedhdu ==> --chdu
+ --wkhdu ==> --whdu
+ --detsnminarea ==> --snminarea
+ --checkdetsn ==> --checksn
+ --detquant ==> --snquant
MakeCatalog:
- Estimation of noise-level is now done per-pixel over the whole
label. Until now the average noise level was used.
+ --objectsfile has been removed. The main input argument is now assumed
+ to be the objects file.
Table:
--column: multiple columns (comma separated) can be used in one
diff --git a/README b/README
index 45e258e..30c0247 100644
--- a/README
+++ b/README
@@ -76,10 +76,13 @@ context under categories/chapters.
Monte Carlo integration. It can also build the profiles on an
over-sampled image.
- - NoiseChisel (astnoisechisel): Detect and segment signal in noise. It
- uses a technique to detect very faint and diffuse, irregularly shaped
- signal in noise (galaxies in the sky), using thresholds that are below
- the Sky value (see arXiv:1505.01664).
+ - NoiseChisel (astnoisechisel): Detect and signal in noise. It uses a
+ technique to detect very faint and diffuse, irregularly shaped signal
+ in noise (galaxies in the sky), using thresholds that are below the Sky
+ value (see arXiv:1505.01664).
+
+ - Segment (astsegment): Segment a detection based on the structure of
+ signal within it.
- Statistics (aststatistics): Get pixel statistics and save histogram and
cumulative frequency plots.
diff --git a/bin/TEMPLATE/Makefile.am b/bin/TEMPLATE/Makefile.am
index f784618..8f159f2 100644
--- a/bin/TEMPLATE/Makefile.am
+++ b/bin/TEMPLATE/Makefile.am
@@ -1,9 +1,9 @@
## Process this file with automake to produce Makefile.inx
##
## Original author:
-## Your Name <address@hidden>
+## Mohammad akhlaghi <address@hidden>
## Contributing author(s):
-## Copyright (C) YYYY, Free Software Foundation, Inc.
+## Copyright (C) 2016-2018, Free Software Foundation, Inc.
##
## Gnuastro is free software: you can redistribute it and/or modify it
## under the terms of the GNU General Public License as published by
diff --git a/bin/TEMPLATE/TEMPLATE.c b/bin/TEMPLATE/TEMPLATE.c
index 60d6340..912b646 100644
--- a/bin/TEMPLATE/TEMPLATE.c
+++ b/bin/TEMPLATE/TEMPLATE.c
@@ -3,9 +3,9 @@ TEMPLATE - A minimal set of files and functions to define a
program.
TEMPLATE is part of GNU Astronomy Utilities (Gnuastro) package.
Original author:
- Your Name <address@hidden>
+ Mohammad akhlaghi <address@hidden>
Contributing author(s):
-Copyright (C) YYYY, Free Software Foundation, Inc.
+Copyright (C) 2016-2018, Free Software Foundation, Inc.
Gnuastro is free software: you can redistribute it and/or modify it
under the terms of the GNU General Public License as published by the
diff --git a/bin/TEMPLATE/TEMPLATE.h b/bin/TEMPLATE/TEMPLATE.h
index 8349e7a..5f5e662 100644
--- a/bin/TEMPLATE/TEMPLATE.h
+++ b/bin/TEMPLATE/TEMPLATE.h
@@ -3,9 +3,9 @@ TEMPLATE - A minimal set of files and functions to define a
program.
TEMPLATE is part of GNU Astronomy Utilities (Gnuastro) package.
Original author:
- Your Name <address@hidden>
+ Mohammad akhlaghi <address@hidden>
Contributing author(s):
-Copyright (C) YYYY, Free Software Foundation, Inc.
+Copyright (C) 2016-2018, Free Software Foundation, Inc.
Gnuastro is free software: you can redistribute it and/or modify it
under the terms of the GNU General Public License as published by the
diff --git a/bin/TEMPLATE/args.h b/bin/TEMPLATE/args.h
index 1b78e62..e4ae3eb 100644
--- a/bin/TEMPLATE/args.h
+++ b/bin/TEMPLATE/args.h
@@ -3,9 +3,9 @@ TEMPLATE - A minimal set of files and functions to define a
program.
TEMPLATE is part of GNU Astronomy Utilities (Gnuastro) package.
Original author:
- Your Name <address@hidden>
+ Mohammad akhlaghi <address@hidden>
Contributing author(s):
-Copyright (C) YYYY, Free Software Foundation, Inc.
+Copyright (C) 2016-2018, Free Software Foundation, Inc.
Gnuastro is free software: you can redistribute it and/or modify it
under the terms of the GNU General Public License as published by the
diff --git a/bin/TEMPLATE/authors-cite.h b/bin/TEMPLATE/authors-cite.h
index 8da944d..607abde 100644
--- a/bin/TEMPLATE/authors-cite.h
+++ b/bin/TEMPLATE/authors-cite.h
@@ -3,9 +3,9 @@ TEMPLATE - A minimal set of files and functions to define a
program.
TEMPLATE is part of GNU Astronomy Utilities (Gnuastro) package.
Original author:
- Your Name <address@hidden>
+ Mohammad akhlaghi <address@hidden>
Contributing author(s):
-Copyright (C) YYYY, Free Software Foundation, Inc.
+Copyright (C) 2016-2018, Free Software Foundation, Inc.
Gnuastro is free software: you can redistribute it and/or modify it
under the terms of the GNU General Public License as published by the
diff --git a/bin/TEMPLATE/main.c b/bin/TEMPLATE/main.c
index aff41a4..a7c450c 100644
--- a/bin/TEMPLATE/main.c
+++ b/bin/TEMPLATE/main.c
@@ -3,9 +3,9 @@ TEMPLATE - A minimal set of files and functions to define a
program.
TEMPLATE is part of GNU Astronomy Utilities (Gnuastro) package.
Original author:
- Your Name <address@hidden>
+ Mohammad akhlaghi <address@hidden>
Contributing author(s):
-Copyright (C) YYYY, Free Software Foundation, Inc.
+Copyright (C) 2016-2018, Free Software Foundation, Inc.
Gnuastro is free software: you can redistribute it and/or modify it
under the terms of the GNU General Public License as published by the
@@ -40,14 +40,14 @@ main (int argc, char *argv[])
struct timeval t1;
struct TEMPLATEparams p={{{0},0},0};
- /* Set they starting time. */
+ /* Set the starting time. */
time(&p.rawtime);
gettimeofday(&t1, NULL);
/* Read the input parameters. */
ui_read_check_inputs_setup(argc, argv, &p);
- /* Run MakeProfiles */
+ /* Run TEMPLATE */
TEMPLATE(&p);
/* Free all non-freed allocations. */
diff --git a/bin/TEMPLATE/main.h b/bin/TEMPLATE/main.h
index ef81396..9a46115 100644
--- a/bin/TEMPLATE/main.h
+++ b/bin/TEMPLATE/main.h
@@ -3,9 +3,9 @@ TEMPLATE - A minimal set of files and functions to define a
program.
TEMPLATE is part of GNU Astronomy Utilities (Gnuastro) package.
Original author:
- Your Name <address@hidden>
+ Mohammad akhlaghi <address@hidden>
Contributing author(s):
-Copyright (C) YYYY, Free Software Foundation, Inc.
+Copyright (C) 2016-2018, Free Software Foundation, Inc.
Gnuastro is free software: you can redistribute it and/or modify it
under the terms of the GNU General Public License as published by the
@@ -43,13 +43,13 @@ along with Gnuastro. If not, see
<http://www.gnu.org/licenses/>.
struct TEMPLATEparams
{
/* From command-line */
- struct gal_options_common_params cp; /* Common parameters. */
- gal_list_str_t *multivalue; /* Pointer to multivalue. */
- char *inputname; /* Input filename. */
- uint8_t *onoff; /* How to store on/off options. */
+ struct gal_options_common_params cp; /* Common parameters. */
+ char *inputname; /* Input filename. */
+ gal_list_str_t *multivalue; /* List of values given to "multivalue" */
+ uint8_t onoff; /* ==1 if option is called, ==0 otherwise. */
/* Output: */
- time_t rawtime; /* Starting time of the program. */
+ time_t rawtime; /* Starting time of the program. */
};
#endif
diff --git a/bin/TEMPLATE/ui.c b/bin/TEMPLATE/ui.c
index 8f11ebe..10a5886 100644
--- a/bin/TEMPLATE/ui.c
+++ b/bin/TEMPLATE/ui.c
@@ -3,9 +3,9 @@ TEMPLATE - A minimal set of files and functions to define a
program.
TEMPLATE is part of GNU Astronomy Utilities (Gnuastro) package.
Original author:
- Your Name <address@hidden>
+ Mohammad akhlaghi <address@hidden>
Contributing author(s):
-Copyright (C) YYYY, Free Software Foundation, Inc.
+Copyright (C) 2016-2018, Free Software Foundation, Inc.
Gnuastro is free software: you can redistribute it and/or modify it
under the terms of the GNU General Public License as published by the
@@ -219,12 +219,15 @@ ui_check_options_and_arguments(struct TEMPLATEparams *p)
a HDU is also given. */
if(p->inputname)
{
+ /* Check if it exists. */
+ gal_checkset_check_file(p->inputname);
+
+ /* If it is FITS, a HDU is also mandatory. */
if( gal_fits_name_is_fits(p->inputname) && p->cp.hdu==NULL )
error(EXIT_FAILURE, 0, "no HDU specified. When the input is a FITS "
"file, a HDU must also be specified, you can use the `--hdu' "
"(`-h') option and give it the HDU number (starting from "
"zero), extension name, or anything acceptable by CFITSIO");
-
}
else
error(EXIT_FAILURE, 0, "no input file is specified");
diff --git a/bin/TEMPLATE/ui.h b/bin/TEMPLATE/ui.h
index 290caa1..163bf42 100644
--- a/bin/TEMPLATE/ui.h
+++ b/bin/TEMPLATE/ui.h
@@ -3,9 +3,9 @@ TEMPLATE - A minimal set of files and functions to define a
program.
TEMPLATE is part of GNU Astronomy Utilities (Gnuastro) package.
Original author:
- Your Name <address@hidden>
+ Mohammad akhlaghi <address@hidden>
Contributing author(s):
-Copyright (C) YYYY, Free Software Foundation, Inc.
+Copyright (C) 2016-2018, Free Software Foundation, Inc.
Gnuastro is free software: you can redistribute it and/or modify it
under the terms of the GNU General Public License as published by the
diff --git a/bin/mkcatalog/Makefile.am b/bin/mkcatalog/Makefile.am
index a929552..3576618 100644
--- a/bin/mkcatalog/Makefile.am
+++ b/bin/mkcatalog/Makefile.am
@@ -30,10 +30,10 @@ bin_PROGRAMS = astmkcatalog
astmkcatalog_LDADD = -lgnuastro
-astmkcatalog_SOURCES = main.c ui.c mkcatalog.c columns.c upperlimit.c
+astmkcatalog_SOURCES = main.c ui.c mkcatalog.c columns.c upperlimit.c parse.c
EXTRA_DIST = main.h authors-cite.h args.h ui.h mkcatalog.h columns.h \
- upperlimit.h
+ upperlimit.h parse.h
diff --git a/bin/mkcatalog/args.h b/bin/mkcatalog/args.h
index b589c91..e6e95f4 100644
--- a/bin/mkcatalog/args.h
+++ b/bin/mkcatalog/args.h
@@ -33,29 +33,29 @@ struct argp_option program_options[] =
{
/* Input options. */
{
- "objectsfile",
- UI_KEY_OBJECTSFILE,
+ "valuesfile",
+ UI_KEY_VALUESFILE,
"STR",
0,
- "Image containing object/detection labels.",
+ "Values/brightness dataset.",
GAL_OPTIONS_GROUP_INPUT,
- &p->objectsfile,
+ &p->valuesfile,
GAL_TYPE_STRING,
GAL_OPTIONS_RANGE_ANY,
GAL_OPTIONS_NOT_MANDATORY,
GAL_OPTIONS_NOT_SET
},
{
- "objectshdu",
- UI_KEY_OBJECTSHDU,
+ "valueshdu",
+ UI_KEY_VALUESHDU,
"STR",
0,
- "Object image extension name or number.",
+ "Name or number of extension containing values.",
GAL_OPTIONS_GROUP_INPUT,
- &p->objectshdu,
+ &p->valueshdu,
GAL_TYPE_STRING,
GAL_OPTIONS_RANGE_ANY,
- GAL_OPTIONS_MANDATORY,
+ GAL_OPTIONS_NOT_MANDATORY,
GAL_OPTIONS_NOT_SET
},
{
@@ -63,7 +63,7 @@ struct argp_option program_options[] =
UI_KEY_CLUMPSFILE,
"STR",
0,
- "Image containing clump labels.",
+ "Dataset containing clump labels.",
GAL_OPTIONS_GROUP_INPUT,
&p->clumpsfile,
GAL_TYPE_STRING,
@@ -76,7 +76,7 @@ struct argp_option program_options[] =
UI_KEY_CLUMPSHDU,
"STR",
0,
- "Clump image extension name or number.",
+ "Clump labels extension name or number.",
GAL_OPTIONS_GROUP_INPUT,
&p->clumpshdu,
GAL_TYPE_STRING,
@@ -107,7 +107,7 @@ struct argp_option program_options[] =
&p->skyhdu,
GAL_TYPE_STRING,
GAL_OPTIONS_RANGE_ANY,
- GAL_OPTIONS_MANDATORY,
+ GAL_OPTIONS_NOT_MANDATORY,
GAL_OPTIONS_NOT_SET
},
{
@@ -128,12 +128,12 @@ struct argp_option program_options[] =
UI_KEY_STDHDU,
"STR",
0,
- "Sky image extension name or number.",
+ "Sky STD extension name or number.",
GAL_OPTIONS_GROUP_INPUT,
&p->stdhdu,
GAL_TYPE_STRING,
GAL_OPTIONS_RANGE_ANY,
- GAL_OPTIONS_MANDATORY,
+ GAL_OPTIONS_NOT_MANDATORY,
GAL_OPTIONS_NOT_SET
},
{
@@ -150,48 +150,48 @@ struct argp_option program_options[] =
GAL_OPTIONS_NOT_SET
},
{
- "skysubtracted",
- UI_KEY_SKYSUBTRACTED,
+ "variance",
+ UI_KEY_VARIANCE,
0,
0,
- "Input is already sky subtracted (for S/N).",
+ "STD input dataset is actually variance.",
GAL_OPTIONS_GROUP_INPUT,
- &p->skysubtracted,
+ &p->variance,
GAL_OPTIONS_NO_ARG_TYPE,
GAL_OPTIONS_RANGE_0_OR_1,
GAL_OPTIONS_NOT_MANDATORY,
GAL_OPTIONS_NOT_SET
},
{
- "threshold",
- UI_KEY_THRESHOLD,
- "FLT",
+ "subtractsky",
+ UI_KEY_SUBTRACTSKY,
+ 0,
0,
- "Use pixels more than this multiple of STD.",
+ "Subtract the Sky dataset from the values.",
GAL_OPTIONS_GROUP_INPUT,
- &p->threshold,
- GAL_TYPE_FLOAT32,
- GAL_OPTIONS_RANGE_ANY,
+ &p->subtractsky,
+ GAL_OPTIONS_NO_ARG_TYPE,
+ GAL_OPTIONS_RANGE_0_OR_1,
GAL_OPTIONS_NOT_MANDATORY,
GAL_OPTIONS_NOT_SET
},
+
+
+
+ /* Output. */
{
- "variance",
- UI_KEY_VARIANCE,
+ "clumpscat",
+ UI_KEY_CLUMPSCAT,
0,
0,
- "Image given as STD is actually variance.",
- GAL_OPTIONS_GROUP_INPUT,
- &p->variance,
+ "Make a clumps catalog also.",
+ GAL_OPTIONS_GROUP_OUTPUT,
+ &p->clumpscat,
GAL_OPTIONS_NO_ARG_TYPE,
GAL_OPTIONS_RANGE_0_OR_1,
GAL_OPTIONS_NOT_MANDATORY,
GAL_OPTIONS_NOT_SET
},
-
-
-
- /* Output. */
{
"sfmagnsigma",
UI_KEY_SFMAGNSIGMA,
@@ -320,6 +320,20 @@ struct argp_option program_options[] =
GAL_OPTIONS_MANDATORY,
GAL_OPTIONS_NOT_SET
},
+ {
+ "checkupperlimit",
+ UI_KEY_CHECKUPPERLIMIT,
+ "INT[,INT]",
+ 0,
+ "Check random distribution for one label.",
+ UI_GROUP_UPPERLIMIT,
+ &p->checkupperlimit,
+ GAL_TYPE_STRING,
+ GAL_OPTIONS_RANGE_GT_0,
+ GAL_OPTIONS_NOT_MANDATORY,
+ GAL_OPTIONS_NOT_SET,
+ ui_check_upperlimit
+ },
@@ -846,7 +860,7 @@ struct argp_option program_options[] =
UI_KEY_UPPERLIMITSIGMA,
0,
0,
- "Place in upperlimit distribution (sigma multiple).",
+ "Place in random distribution (sigma multiple).",
UI_GROUP_COLUMNS_BRIGHTNESS,
0,
GAL_TYPE_INVALID,
@@ -982,7 +996,7 @@ struct argp_option program_options[] =
UI_KEY_AREA,
0,
0,
- "Number of pixels in clump or object.",
+ "Number of non-blank valued pixels.",
UI_GROUP_COLUMNS_MORPHOLOGY,
0,
GAL_TYPE_INVALID,
@@ -996,7 +1010,7 @@ struct argp_option program_options[] =
UI_KEY_CLUMPSAREA,
0,
0,
- "Sum of all clump areas in an object.",
+ "Non-blank area covered by clumps.",
UI_GROUP_COLUMNS_MORPHOLOGY,
0,
GAL_TYPE_INVALID,
@@ -1010,7 +1024,21 @@ struct argp_option program_options[] =
UI_KEY_WEIGHTAREA,
0,
0,
- "Area used for flux weighted positions.",
+ "Area used for value weighted positions.",
+ UI_GROUP_COLUMNS_MORPHOLOGY,
+ 0,
+ GAL_TYPE_INVALID,
+ GAL_OPTIONS_RANGE_ANY,
+ GAL_OPTIONS_NOT_MANDATORY,
+ GAL_OPTIONS_NOT_SET,
+ ui_column_codes_ll
+ },
+ {
+ "geoarea",
+ UI_KEY_GEOAREA,
+ 0,
+ 0,
+ "Area labled region (irrespective of value).",
UI_GROUP_COLUMNS_MORPHOLOGY,
0,
GAL_TYPE_INVALID,
diff --git a/bin/mkcatalog/astmkcatalog.conf b/bin/mkcatalog/astmkcatalog.conf
index b2d111f..79a9ce2 100644
--- a/bin/mkcatalog/astmkcatalog.conf
+++ b/bin/mkcatalog/astmkcatalog.conf
@@ -18,13 +18,12 @@
# warranty.
# Input:
- hdu 1
- objectshdu 2
- clumpshdu 3
+ hdu OBJECTS
+ valueshdu 1
+ clumpshdu CLUMPS
skyhdu SKY
stdhdu SKY_STD
zeropoint 0.0
- skysubtracted 0
# Output:
sfmagnsigma 1
diff --git a/bin/mkcatalog/columns.c b/bin/mkcatalog/columns.c
index f5e075f..188b9ce 100644
--- a/bin/mkcatalog/columns.c
+++ b/bin/mkcatalog/columns.c
@@ -59,14 +59,14 @@ columns_alloc_radec(struct mkcatalogparams *p)
/* For objects. */
if(p->wcs_vo==NULL)
- for(i=0;i<p->input->ndim;++i)
+ for(i=0;i<p->objects->ndim;++i)
gal_list_data_add_alloc(&p->wcs_vo, NULL, GAL_TYPE_FLOAT64, 1,
&p->numobjects, NULL, 0, p->cp.minmapsize,
NULL, NULL, NULL);
/* For clumps */
if(p->clumps && p->wcs_vc==NULL)
- for(i=0;i<p->input->ndim;++i)
+ for(i=0;i<p->objects->ndim;++i)
gal_list_data_add_alloc(&p->wcs_vc, NULL, GAL_TYPE_FLOAT64, 1,
&p->numclumps, NULL, 0, p->cp.minmapsize,
NULL, NULL, NULL);
@@ -84,14 +84,14 @@ columns_alloc_georadec(struct mkcatalogparams *p)
/* For objects. */
if(p->wcs_go==NULL)
- for(i=0;i<p->input->ndim;++i)
+ for(i=0;i<p->objects->ndim;++i)
gal_list_data_add_alloc(&p->wcs_go, NULL, GAL_TYPE_FLOAT64, 1,
&p->numobjects, NULL, 0, p->cp.minmapsize,
NULL, NULL, NULL);
/* For clumps */
if(p->clumps && p->wcs_gc==NULL)
- for(i=0;i<p->input->ndim;++i)
+ for(i=0;i<p->objects->ndim;++i)
gal_list_data_add_alloc(&p->wcs_gc, NULL, GAL_TYPE_FLOAT64, 1,
&p->numclumps, NULL, 0, p->cp.minmapsize,
NULL, NULL, NULL);
@@ -108,7 +108,7 @@ columns_alloc_clumpsradec(struct mkcatalogparams *p)
size_t i;
if(p->wcs_vcc==NULL)
- for(i=0;i<p->input->ndim;++i)
+ for(i=0;i<p->objects->ndim;++i)
gal_list_data_add_alloc(&p->wcs_vcc, NULL, GAL_TYPE_FLOAT64, 1,
&p->numobjects, NULL, 0, p->cp.minmapsize,
NULL, NULL, NULL);
@@ -125,7 +125,7 @@ columns_alloc_clumpsgeoradec(struct mkcatalogparams *p)
size_t i;
if(p->wcs_gcc==NULL)
- for(i=0;i<p->input->ndim;++i)
+ for(i=0;i<p->objects->ndim;++i)
gal_list_data_add_alloc(&p->wcs_gcc, NULL, GAL_TYPE_FLOAT64, 1,
&p->numobjects, NULL, 0, p->cp.minmapsize,
NULL, NULL, NULL);
@@ -139,10 +139,10 @@ columns_alloc_clumpsgeoradec(struct mkcatalogparams *p)
#define SET_WCS_PREPARE(ARR, LIST, ARRNAME) { \
d=0; \
errno=0; \
- (ARR)=malloc(p->input->ndim * sizeof (ARR) ); \
+ (ARR)=malloc(p->objects->ndim * sizeof (ARR) ); \
if( (ARR)==NULL ) \
error(EXIT_FAILURE, 0, "%s: %zu bytes for %s", __func__, \
- p->input->ndim * sizeof (ARR), ARRNAME); \
+ p->objects->ndim * sizeof (ARR), ARRNAME); \
for(tmp=(LIST);tmp!=NULL;tmp=tmp->next) (ARR)[d++]=tmp->array; \
}
@@ -211,13 +211,13 @@ columns_wcs_preparation(struct mkcatalogparams *p)
case UI_KEY_CLUMPSW2:
case UI_KEY_CLUMPSGEOW1:
case UI_KEY_CLUMPSGEOW2:
- if(p->input->wcs)
+ if(p->objects->wcs)
continue_wcs_check=0;
else
- error(EXIT_FAILURE, 0, "input doesn't have WCS meta-data for "
- "defining world coordinates (like RA and Dec). Atleast "
- "one of the requested columns requires this "
- "information");
+ error(EXIT_FAILURE, 0, "%s (hdu: %s): no WCS meta-data "
+ "found by WCSLIB. Atleast one of the requested columns "
+ "requires world coordinate system meta-data",
+ p->objectsfile, p->cp.hdu);
break;
}
}
@@ -232,7 +232,7 @@ columns_wcs_preparation(struct mkcatalogparams *p)
case UI_KEY_RA:
case UI_KEY_DEC:
/* Check all the CTYPES. */
- for(i=0;i<p->input->ndim;++i)
+ for(i=0;i<p->objects->ndim;++i)
if( !strcmp(p->ctype[i], colcode->v==UI_KEY_RA ? "RA" : "DEC") )
{
colcode->v = i==0 ? UI_KEY_W1 : UI_KEY_W2;
@@ -240,9 +240,9 @@ columns_wcs_preparation(struct mkcatalogparams *p)
}
/* Make sure it actually existed. */
- if(i==p->input->ndim)
+ if(i==p->objects->ndim)
error(EXIT_FAILURE, 0, "%s (hdu: %s): %s not present in any of "
- "the WCS axis types (CTYPE)", p->inputname, p->cp.hdu,
+ "the WCS axis types (CTYPE)", p->objectsfile, p->cp.hdu,
colcode->v==UI_KEY_RA ? "RA" : "DEC");
break;
}
@@ -294,7 +294,7 @@ columns_define_alloc(struct mkcatalogparams *p)
unit = "counter";
ocomment = "Object identifier.";
ccomment = NULL;
- otype = GAL_TYPE_INT32; /* Same type as clumps image. */
+ otype = GAL_TYPE_INT32;
ctype = GAL_TYPE_INVALID;
disp_fmt = 0;
disp_width = 6;
@@ -344,42 +344,53 @@ columns_define_alloc(struct mkcatalogparams *p)
case UI_KEY_AREA:
name = "AREA";
unit = "counter";
- ocomment = "Number of pixels covered.";
+ ocomment = "Number of non-blank pixels.";
ccomment = ocomment;
otype = GAL_TYPE_INT32;
ctype = GAL_TYPE_INT32;
disp_fmt = 0;
- disp_width = 5;
+ disp_width = 6;
disp_precision = 0;
- oiflag[ OCOL_NUMALL ] = 1;
- ciflag[ CCOL_NUMALL ] = 1;
+ oiflag[ OCOL_NUM ] = ciflag[ CCOL_NUM ] = 1;
break;
case UI_KEY_CLUMPSAREA:
- name = "CLUMPS_AREA";
+ name = "AREA_CLUMPS";
unit = "counter";
ocomment = "Total number of clump pixels in object.";
ccomment = NULL;
otype = GAL_TYPE_INT32;
ctype = GAL_TYPE_INVALID;
disp_fmt = 0;
- disp_width = 5;
+ disp_width = 6;
disp_precision = 0;
oiflag[ OCOL_C_NUM ] = 1;
break;
case UI_KEY_WEIGHTAREA:
- name = "WEIGHT_AREA";
+ name = "AREA_WEIGHT";
unit = "counter";
ocomment = "Area used for flux-weighted positions.";
ccomment = ocomment;
otype = GAL_TYPE_INT32;
ctype = GAL_TYPE_INT32;
disp_fmt = 0;
- disp_width = 5;
+ disp_width = 6;
+ disp_precision = 0;
+ oiflag[ OCOL_NUMWHT ] = ciflag[ CCOL_NUMWHT ] = 1;
+ break;
+
+ case UI_KEY_GEOAREA:
+ name = "AREA_FULL";
+ unit = "counter";
+ ocomment = "Full area of label (irrespective of values).";
+ ccomment = ocomment;
+ otype = GAL_TYPE_INT32;
+ ctype = GAL_TYPE_INT32;
+ disp_fmt = 0;
+ disp_width = 6;
disp_precision = 0;
- oiflag[ OCOL_NUMWHT ] = 1;
- ciflag[ CCOL_NUMWHT ] = 1;
+ oiflag[ OCOL_NUMALL ] = ciflag[ CCOL_NUMALL ] = 1;
break;
case UI_KEY_X:
@@ -392,8 +403,10 @@ columns_define_alloc(struct mkcatalogparams *p)
disp_fmt = GAL_TABLE_DISPLAY_FMT_FLOAT;
disp_width = 10;
disp_precision = 3;
- oiflag[ OCOL_VX ] = 1;
- ciflag[ CCOL_VX ] = 1;
+ oiflag[ OCOL_VX ] = ciflag[ CCOL_VX ] = 1;
+ oiflag[ OCOL_GX ] = ciflag[ CCOL_GX ] = 1;
+ oiflag[ OCOL_SUMWHT ] = ciflag[ CCOL_SUMWHT ] = 1;
+ oiflag[ OCOL_NUMWHT ] = ciflag[ CCOL_NUMWHT ] = 1;
break;
case UI_KEY_Y:
@@ -406,8 +419,10 @@ columns_define_alloc(struct mkcatalogparams *p)
disp_fmt = GAL_TABLE_DISPLAY_FMT_FLOAT;
disp_width = 10;
disp_precision = 3;
- oiflag[ OCOL_VY ] = 1;
- ciflag[ CCOL_VY ] = 1;
+ oiflag[ OCOL_VY ] = ciflag[ CCOL_VY ] = 1;
+ oiflag[ OCOL_GY ] = ciflag[ CCOL_GY ] = 1;
+ oiflag[ OCOL_SUMWHT ] = ciflag[ CCOL_SUMWHT ] = 1;
+ oiflag[ OCOL_NUMWHT ] = ciflag[ CCOL_NUMWHT ] = 1;
break;
case UI_KEY_GEOX:
@@ -420,8 +435,8 @@ columns_define_alloc(struct mkcatalogparams *p)
disp_fmt = GAL_TABLE_DISPLAY_FMT_FLOAT;
disp_width = 10;
disp_precision = 3;
- oiflag[ OCOL_GX ] = 1;
- ciflag[ CCOL_GX ] = 1;
+ oiflag[ OCOL_GX ] = ciflag[ CCOL_GX ] = 1;
+ oiflag[ OCOL_NUMALL ] = ciflag[ CCOL_NUMALL ] = 1;
break;
case UI_KEY_GEOY:
@@ -434,8 +449,8 @@ columns_define_alloc(struct mkcatalogparams *p)
disp_fmt = GAL_TABLE_DISPLAY_FMT_FLOAT;
disp_width = 10;
disp_precision = 3;
- oiflag[ OCOL_GY ] = 1;
- ciflag[ CCOL_GY ] = 1;
+ oiflag[ OCOL_GY ] = ciflag[ CCOL_GY ] = 1;
+ oiflag[ OCOL_NUMALL ] = ciflag[ CCOL_NUMALL ] = 1;
break;
case UI_KEY_CLUMPSX:
@@ -448,7 +463,10 @@ columns_define_alloc(struct mkcatalogparams *p)
disp_fmt = GAL_TABLE_DISPLAY_FMT_FLOAT;
disp_width = 10;
disp_precision = 3;
- oiflag[ OCOL_C_VX ] = 1;
+ oiflag[ OCOL_C_VX ] = 1;
+ oiflag[ OCOL_C_GX ] = 1;
+ oiflag[ OCOL_C_SUMWHT ] = 1;
+ oiflag[ OCOL_C_NUMWHT ] = 1;
break;
case UI_KEY_CLUMPSY:
@@ -461,7 +479,10 @@ columns_define_alloc(struct mkcatalogparams *p)
disp_fmt = GAL_TABLE_DISPLAY_FMT_FLOAT;
disp_width = 10;
disp_precision = 3;
- oiflag[ OCOL_C_VY ] = 1;
+ oiflag[ OCOL_C_VY ] = 1;
+ oiflag[ OCOL_C_GY ] = 1;
+ oiflag[ OCOL_C_SUMWHT ] = 1;
+ oiflag[ OCOL_C_NUMWHT ] = 1;
break;
case UI_KEY_CLUMPSGEOX:
@@ -474,7 +495,8 @@ columns_define_alloc(struct mkcatalogparams *p)
disp_fmt = GAL_TABLE_DISPLAY_FMT_FLOAT;
disp_width = 10;
disp_precision = 3;
- oiflag[ OCOL_C_GX ] = 1;
+ oiflag[ OCOL_C_GX ] = 1;
+ oiflag[ OCOL_C_NUMALL ] = 1;
break;
case UI_KEY_CLUMPSGEOY:
@@ -487,12 +509,13 @@ columns_define_alloc(struct mkcatalogparams *p)
disp_fmt = GAL_TABLE_DISPLAY_FMT_FLOAT;
disp_width = 10;
disp_precision = 3;
- oiflag[ OCOL_C_GY ] = 1;
+ oiflag[ OCOL_C_GY ] = 1;
+ oiflag[ OCOL_C_NUMALL ] = 1;
break;
case UI_KEY_W1:
name = p->ctype[0];
- unit = p->input->wcs->cunit[0];
+ unit = p->objects->wcs->cunit[0];
ocomment = "Flux weighted center (WCS axis 1).";
ccomment = ocomment;
otype = GAL_TYPE_FLOAT64;
@@ -500,16 +523,18 @@ columns_define_alloc(struct mkcatalogparams *p)
disp_fmt = GAL_TABLE_DISPLAY_FMT_FLOAT;
disp_width = 13;
disp_precision = 7;
- oiflag[ OCOL_VX ] = 1;
- oiflag[ OCOL_VY ] = 1;
- oiflag[ CCOL_VX ] = 1;
- oiflag[ CCOL_VY ] = 1;
columns_alloc_radec(p);
+ oiflag[ OCOL_VX ] = ciflag[ CCOL_VX ] = 1;
+ oiflag[ OCOL_GX ] = ciflag[ CCOL_GX ] = 1;
+ oiflag[ OCOL_VY ] = ciflag[ CCOL_VY ] = 1;
+ oiflag[ OCOL_GY ] = ciflag[ CCOL_GY ] = 1;
+ oiflag[ OCOL_SUMWHT ] = ciflag[ CCOL_SUMWHT ] = 1;
+ oiflag[ OCOL_NUMALL ] = ciflag[ CCOL_NUMALL ] = 1;
break;
case UI_KEY_W2:
name = p->ctype[1];
- unit = p->input->wcs->cunit[1];
+ unit = p->objects->wcs->cunit[1];
ocomment = "Flux weighted center (WCS axis 2).";
ccomment = ocomment;
otype = GAL_TYPE_FLOAT64;
@@ -517,16 +542,18 @@ columns_define_alloc(struct mkcatalogparams *p)
disp_fmt = GAL_TABLE_DISPLAY_FMT_FLOAT;
disp_width = 13;
disp_precision = 7;
- oiflag[ OCOL_VX ] = 1;
- oiflag[ OCOL_VY ] = 1;
- oiflag[ CCOL_VX ] = 1;
- oiflag[ CCOL_VY ] = 1;
columns_alloc_radec(p);
+ oiflag[ OCOL_VX ] = ciflag[ CCOL_VX ] = 1;
+ oiflag[ OCOL_GX ] = ciflag[ CCOL_GX ] = 1;
+ oiflag[ OCOL_VY ] = ciflag[ CCOL_VY ] = 1;
+ oiflag[ OCOL_GY ] = ciflag[ CCOL_GY ] = 1;
+ oiflag[ OCOL_SUMWHT ] = ciflag[ CCOL_SUMWHT ] = 1;
+ oiflag[ OCOL_NUMALL ] = ciflag[ CCOL_NUMALL ] = 1;
break;
case UI_KEY_GEOW1:
name = gal_checkset_malloc_cat("GEO_", p->ctype[0]);
- unit = p->input->wcs->cunit[0];
+ unit = p->objects->wcs->cunit[0];
ocomment = "Geometric center (WCS axis 1).";
ccomment = ocomment;
otype = GAL_TYPE_FLOAT64;
@@ -534,16 +561,15 @@ columns_define_alloc(struct mkcatalogparams *p)
disp_fmt = GAL_TABLE_DISPLAY_FMT_FLOAT;
disp_width = 13;
disp_precision = 7;
- oiflag[ OCOL_GX ] = 1;
- oiflag[ OCOL_GY ] = 1;
- ciflag[ CCOL_GX ] = 1;
- ciflag[ CCOL_GY ] = 1;
columns_alloc_georadec(p);
+ oiflag[ OCOL_GX ] = ciflag[ CCOL_GX ] = 1;
+ oiflag[ OCOL_GY ] = ciflag[ CCOL_GY ] = 1;
+ oiflag[ OCOL_NUMALL ] = ciflag[ CCOL_NUMALL ] = 1;
break;
case UI_KEY_GEOW2:
name = gal_checkset_malloc_cat("GEO_", p->ctype[1]);
- unit = p->input->wcs->cunit[1];
+ unit = p->objects->wcs->cunit[1];
ocomment = "Geometric center (WCS axis 2).";
ccomment = ocomment;
otype = GAL_TYPE_FLOAT64;
@@ -551,16 +577,15 @@ columns_define_alloc(struct mkcatalogparams *p)
disp_fmt = GAL_TABLE_DISPLAY_FMT_FLOAT;
disp_width = 13;
disp_precision = 7;
- oiflag[ OCOL_GX ] = 1;
- oiflag[ OCOL_GY ] = 1;
- ciflag[ CCOL_GX ] = 1;
- ciflag[ CCOL_GY ] = 1;
columns_alloc_georadec(p);
+ oiflag[ OCOL_GX ] = ciflag[ CCOL_GX ] = 1;
+ oiflag[ OCOL_GY ] = ciflag[ CCOL_GY ] = 1;
+ oiflag[ OCOL_NUMALL ] = ciflag[ CCOL_NUMALL ] = 1;
break;
case UI_KEY_CLUMPSW1:
name = gal_checkset_malloc_cat("CLUMPS_", p->ctype[0]);
- unit = p->input->wcs->cunit[0];
+ unit = p->objects->wcs->cunit[0];
ocomment = "Flux.wht center of all clumps (WCS axis 1).";
ccomment = NULL;
otype = GAL_TYPE_FLOAT64;
@@ -568,14 +593,18 @@ columns_define_alloc(struct mkcatalogparams *p)
disp_fmt = GAL_TABLE_DISPLAY_FMT_FLOAT;
disp_width = 13;
disp_precision = 7;
- oiflag[ OCOL_C_VX ] = 1;
- oiflag[ OCOL_C_VY ] = 1;
columns_alloc_clumpsradec(p);
+ oiflag[ OCOL_C_VX ] = 1;
+ oiflag[ OCOL_C_VY ] = 1;
+ oiflag[ OCOL_C_GX ] = 1;
+ oiflag[ OCOL_C_GY ] = 1;
+ oiflag[ OCOL_C_SUMWHT ] = 1;
+ oiflag[ OCOL_C_NUMALL ] = 1;
break;
case UI_KEY_CLUMPSW2:
name = gal_checkset_malloc_cat("CLUMPS_", p->ctype[1]);
- unit = p->input->wcs->cunit[1];
+ unit = p->objects->wcs->cunit[1];
ocomment = "Flux.wht center of all clumps (WCS axis 2).";
ccomment = NULL;
otype = GAL_TYPE_FLOAT64;
@@ -583,14 +612,18 @@ columns_define_alloc(struct mkcatalogparams *p)
disp_fmt = GAL_TABLE_DISPLAY_FMT_FLOAT;
disp_width = 15;
disp_precision = 7;
- oiflag[ OCOL_C_VX ] = 1;
- oiflag[ OCOL_C_VY ] = 1;
columns_alloc_clumpsradec(p);
+ oiflag[ OCOL_C_VX ] = 1;
+ oiflag[ OCOL_C_VY ] = 1;
+ oiflag[ OCOL_C_GX ] = 1;
+ oiflag[ OCOL_C_GY ] = 1;
+ oiflag[ OCOL_C_SUMWHT ] = 1;
+ oiflag[ OCOL_C_NUMALL ] = 1;
break;
case UI_KEY_CLUMPSGEOW1:
name = gal_checkset_malloc_cat("CLUMPS_GEO", p->ctype[0]);
- unit = p->input->wcs->cunit[0];
+ unit = p->objects->wcs->cunit[0];
ocomment = "Geometric center of all clumps (WCS axis 1).";
ccomment = NULL;
otype = GAL_TYPE_FLOAT64;
@@ -598,14 +631,15 @@ columns_define_alloc(struct mkcatalogparams *p)
disp_fmt = GAL_TABLE_DISPLAY_FMT_FLOAT;
disp_width = 13;
disp_precision = 7;
- oiflag[ OCOL_C_GX ] = 1;
- oiflag[ OCOL_C_GY ] = 1;
columns_alloc_clumpsgeoradec(p);
+ oiflag[ OCOL_C_GX ] = 1;
+ oiflag[ OCOL_C_GY ] = 1;
+ oiflag[ OCOL_C_NUMALL ] = 1;
break;
case UI_KEY_CLUMPSGEOW2:
name = gal_checkset_malloc_cat("CLUMPS_GEO", p->ctype[1]);
- unit = p->input->wcs->cunit[1];
+ unit = p->objects->wcs->cunit[1];
ocomment = "Geometric center of all clumps (WCS axis 2).";
ccomment = NULL;
otype = GAL_TYPE_FLOAT64;
@@ -613,102 +647,102 @@ columns_define_alloc(struct mkcatalogparams *p)
disp_fmt = GAL_TABLE_DISPLAY_FMT_FLOAT;
disp_width = 13;
disp_precision = 7;
- oiflag[ OCOL_C_GX ] = 1;
- oiflag[ OCOL_C_GY ] = 1;
columns_alloc_clumpsgeoradec(p);
+ oiflag[ OCOL_C_GX ] = 1;
+ oiflag[ OCOL_C_GY ] = 1;
+ oiflag[ OCOL_C_NUMALL ] = 1;
break;
case UI_KEY_BRIGHTNESS:
name = "BRIGHTNESS";
- unit = p->input->unit ? p->input->unit : "pixelunit";
+ unit = MKCATALOG_NO_UNIT;
ocomment = "Brightness (sum of sky subtracted values).";
ccomment = "Brightness (sum of pixels subtracted by rivers).";
otype = GAL_TYPE_FLOAT32;
ctype = GAL_TYPE_FLOAT32;
disp_fmt = GAL_TABLE_DISPLAY_FMT_GENERAL;
disp_width = 10;
- disp_precision = 4;
- oiflag[ OCOL_SUM ] = 1;
- ciflag[ CCOL_SUM ] = 1;
- ciflag[ CCOL_RIV_NUM ] = 1;
- ciflag[ CCOL_RIV_SUM ] = 1;
+ disp_precision = 5;
+ oiflag[ OCOL_NUM ] = ciflag[ CCOL_NUM ] = 1;
+ oiflag[ OCOL_SUM ] = ciflag[ CCOL_SUM ] = 1;
+ ciflag[ CCOL_RIV_NUM ] = 1;
+ ciflag[ CCOL_RIV_SUM ] = 1;
break;
case UI_KEY_BRIGHTNESSERR:
name = "BRIGHTNESS_ERROR";
- unit = p->input->unit ? p->input->unit : "pixelunit";
+ unit = MKCATALOG_NO_UNIT;
ocomment = "Error (1-sigma) in measuring brightness.";
ccomment = ocomment;
otype = GAL_TYPE_FLOAT32;
ctype = GAL_TYPE_FLOAT32;
disp_fmt = GAL_TABLE_DISPLAY_FMT_GENERAL;
disp_width = 10;
- disp_precision = 4;
- oiflag[ OCOL_SUM_VAR ] = 1;
- ciflag[ CCOL_SUM_VAR ] = 1;
- ciflag[ CCOL_RIV_SUM_VAR ] = 1;
+ disp_precision = 5;
+ oiflag[ OCOL_NUM ] = ciflag[ CCOL_NUM ] = 1;
+ oiflag[ OCOL_SUM_VAR ] = ciflag[ CCOL_SUM_VAR ] = 1;
+ ciflag[ CCOL_RIV_NUM ] = 1;
+ ciflag[ CCOL_RIV_SUM_VAR ] = 1;
break;
case UI_KEY_CLUMPSBRIGHTNESS:
name = "CLUMPS_BRIGHTNESS";
- unit = p->input->unit ? p->input->unit : "pixelunit";
+ unit = MKCATALOG_NO_UNIT;
ocomment = "Brightness (sum of pixel values) in clumps.";
ccomment = NULL;
otype = GAL_TYPE_FLOAT32;
ctype = GAL_TYPE_INVALID;
disp_fmt = GAL_TABLE_DISPLAY_FMT_GENERAL;
disp_width = 10;
- disp_precision = 4;
+ disp_precision = 5;
+ oiflag[ OCOL_C_NUM ] = 1;
oiflag[ OCOL_C_SUM ] = 1;
break;
case UI_KEY_NORIVERBRIGHTNESS:
name = "NO_RIVER_BRIGHTNESS";
- unit = p->input->unit ? p->input->unit : "pixelunit";
+ unit = MKCATALOG_NO_UNIT;
ocomment = NULL;
ccomment = "Brightness (sum of sky subtracted values).";
otype = GAL_TYPE_INVALID;
ctype = GAL_TYPE_FLOAT32;
disp_fmt = GAL_TABLE_DISPLAY_FMT_GENERAL;
disp_width = 10;
- disp_precision = 4;
+ disp_precision = 5;
+ ciflag[ CCOL_NUM ] = 1;
ciflag[ CCOL_SUM ] = 1;
break;
case UI_KEY_MEAN:
name = "MEAN";
- unit = p->input->unit ? p->input->unit : "pixelunit";
+ unit = MKCATALOG_NO_UNIT;
ocomment = "Mean of sky subtracted values.";
ccomment = "Mean of pixels subtracted by rivers.";
otype = GAL_TYPE_FLOAT32;
ctype = GAL_TYPE_FLOAT32;
disp_fmt = GAL_TABLE_DISPLAY_FMT_GENERAL;
disp_width = 10;
- disp_precision = 4;
- oiflag[ OCOL_NUM ] = 1;
- oiflag[ OCOL_SUM ] = 1;
- ciflag[ CCOL_NUM ] = 1;
- ciflag[ CCOL_SUM ] = 1;
- ciflag[ CCOL_RIV_NUM ] = 1;
- ciflag[ CCOL_RIV_SUM ] = 1;
+ disp_precision = 5;
+ oiflag[ OCOL_NUM ] = ciflag[ CCOL_NUM ] = 1;
+ oiflag[ OCOL_SUM ] = ciflag[ CCOL_SUM ] = 1;
+ ciflag[ CCOL_RIV_NUM ] = 1;
+ ciflag[ CCOL_RIV_SUM ] = 1;
break;
case UI_KEY_MEDIAN:
name = "MEDIAN";
- unit = p->input->unit ? p->input->unit : "pixelunit";
+ unit = MKCATALOG_NO_UNIT;
ocomment = "Median of sky subtracted values.";
ccomment = "Median of pixels subtracted by rivers.";
otype = GAL_TYPE_FLOAT32;
ctype = GAL_TYPE_FLOAT32;
disp_fmt = GAL_TABLE_DISPLAY_FMT_GENERAL;
disp_width = 10;
- disp_precision = 4;
- oiflag[ OCOL_MEDIAN ] = 1;
- oiflag[ OCOL_NUMALL ] = 1;
- ciflag[ CCOL_MEDIAN ] = 1;
- ciflag[ CCOL_NUMALL ] = 1;
- ciflag[ CCOL_RIV_NUM ] = 1;
- ciflag[ CCOL_RIV_SUM ] = 1;
+ disp_precision = 5;
+ oiflag[ OCOL_NUM ] = ciflag[ CCOL_NUM ] = 1;
+ oiflag[ OCOL_MEDIAN ] = ciflag[ CCOL_MEDIAN ] = 1;
+ ciflag[ CCOL_RIV_NUM ] = 1;
+ ciflag[ CCOL_RIV_SUM ] = 1;
break;
case UI_KEY_MAGNITUDE:
@@ -721,9 +755,13 @@ columns_define_alloc(struct mkcatalogparams *p)
disp_fmt = GAL_TABLE_DISPLAY_FMT_FLOAT;
disp_width = 8;
disp_precision = 3;
- oiflag[ OCOL_SUM ] = 1;
- ciflag[ CCOL_SUM ] = 1;
p->hasmag = 1;
+ oiflag[ OCOL_NUM ] = ciflag[ CCOL_NUM ] = 1;
+ oiflag[ OCOL_SUM ] = ciflag[ CCOL_SUM ] = 1;
+ ciflag[ CCOL_SUM ] = 1;
+ ciflag[ CCOL_RIV_NUM ] = 1;
+ ciflag[ CCOL_RIV_SUM ] = 1;
+ ciflag[ CCOL_RIV_NUM ] = 1;
break;
case UI_KEY_MAGNITUDEERR:
@@ -736,12 +774,10 @@ columns_define_alloc(struct mkcatalogparams *p)
disp_fmt = GAL_TABLE_DISPLAY_FMT_FLOAT;
disp_width = 8;
disp_precision = 3;
- oiflag[ OCOL_SUM ] = 1;
- oiflag[ OCOL_SUM_VAR ] = 1;
- ciflag[ CCOL_SUM ] = 1;
- ciflag[ CCOL_SUM_VAR ] = 1;
- ciflag[ CCOL_RIV_SUM ] = 1;
- ciflag[ CCOL_RIV_SUM_VAR ] = 1;
+ oiflag[ OCOL_SUM ] = ciflag[ CCOL_SUM ] = 1;
+ oiflag[ OCOL_SUM_VAR ] = ciflag[ CCOL_SUM_VAR ] = 1;
+ ciflag[ CCOL_RIV_SUM ] = 1;
+ ciflag[ CCOL_RIV_SUM_VAR ] = 1;
break;
case UI_KEY_CLUMPSMAGNITUDE:
@@ -754,21 +790,27 @@ columns_define_alloc(struct mkcatalogparams *p)
disp_fmt = GAL_TABLE_DISPLAY_FMT_FLOAT;
disp_width = 8;
disp_precision = 3;
- oiflag[ OCOL_C_SUM ] = 1;
p->hasmag = 1;
+ oiflag[ OCOL_SUM ] = ciflag[ CCOL_SUM ] = 1;
+ oiflag[ OCOL_SUM_VAR ] = ciflag[ CCOL_SUM_VAR ] = 1;
+ ciflag[ CCOL_RIV_NUM ] = 1;
+ ciflag[ CCOL_RIV_SUM ] = 1;
+ ciflag[ CCOL_RIV_SUM_VAR ] = 1;
break;
case UI_KEY_UPPERLIMIT:
name = "UPPERLIMIT";
- unit = p->input->unit;
+ unit = MKCATALOG_NO_UNIT;
ocomment = "Upper limit value (random positionings).";
ccomment = ocomment;
otype = GAL_TYPE_FLOAT32;
ctype = GAL_TYPE_FLOAT32;
disp_fmt = GAL_TABLE_DISPLAY_FMT_GENERAL;
- disp_width = 8;
- disp_precision = 3;
- p->upperlimit = 1; /* Doesn't need per-pixel calculations. */
+ disp_width = 10;
+ disp_precision = 5;
+ p->upperlimit = 1;
+ oiflag[ OCOL_UPPERLIMIT_B ] = ciflag[ CCOL_UPPERLIMIT_B ] = 1;
+
break;
case UI_KEY_UPPERLIMITMAG:
@@ -781,34 +823,42 @@ columns_define_alloc(struct mkcatalogparams *p)
disp_fmt = GAL_TABLE_DISPLAY_FMT_FLOAT;
disp_width = 8;
disp_precision = 3;
+ p->hasmag = 1;
p->upperlimit = 1;
- p->hasmag = 1; /* Doesn't need per-pixel calculations. */
+ oiflag[ OCOL_UPPERLIMIT_B ] = ciflag[ CCOL_UPPERLIMIT_B ] = 1;
break;
case UI_KEY_UPPERLIMITONESIGMA:
name = "UPPERLIMIT_ONE_SIGMA";
- unit = p->input->unit;
+ unit = MKCATALOG_NO_UNIT;
ocomment = "One sigma value of all random measurements.";
ccomment = ocomment;
otype = GAL_TYPE_FLOAT32;
ctype = GAL_TYPE_FLOAT32;
disp_fmt = GAL_TABLE_DISPLAY_FMT_GENERAL;
- disp_width = 8;
- disp_precision = 3;
+ disp_width = 10;
+ disp_precision = 5;
p->upperlimit = 1;
+ oiflag[ OCOL_UPPERLIMIT_S ] = ciflag[ CCOL_UPPERLIMIT_S ] = 1;
break;
case UI_KEY_UPPERLIMITSIGMA:
name = "UPPERLIMIT_SIGMA";
unit = "frac";
- ocomment = "Place in upperlimit distribution (sigma
multiple).";
+ ocomment = "Place in upperlimit distribution (sigma "
+ "multiple).";
ccomment = ocomment;
otype = GAL_TYPE_FLOAT32;
ctype = GAL_TYPE_FLOAT32;
disp_fmt = GAL_TABLE_DISPLAY_FMT_GENERAL;
- disp_width = 8;
- disp_precision = 3;
+ disp_width = 10;
+ disp_precision = 5;
p->upperlimit = 1;
+ oiflag[ OCOL_NUM ] = ciflag[ CCOL_NUM ] = 1;
+ oiflag[ OCOL_SUM ] = ciflag[ CCOL_SUM ] = 1;
+ oiflag[ OCOL_UPPERLIMIT_S ] = ciflag[ CCOL_UPPERLIMIT_S ] = 1;
+ ciflag[ CCOL_RIV_NUM ] = 1;
+ ciflag[ CCOL_RIV_SUM ] = 1;
break;
case UI_KEY_UPPERLIMITQUANTILE:
@@ -819,9 +869,10 @@ columns_define_alloc(struct mkcatalogparams *p)
otype = GAL_TYPE_FLOAT32;
ctype = GAL_TYPE_FLOAT32;
disp_fmt = GAL_TABLE_DISPLAY_FMT_GENERAL;
- disp_width = 8;
- disp_precision = 3;
+ disp_width = 10;
+ disp_precision = 5;
p->upperlimit = 1;
+ oiflag[ OCOL_UPPERLIMIT_Q ] = ciflag[ CCOL_UPPERLIMIT_Q ] = 1;
break;
case UI_KEY_UPPERLIMITSKEW:
@@ -831,24 +882,24 @@ columns_define_alloc(struct mkcatalogparams *p)
ccomment = ocomment;
otype = GAL_TYPE_FLOAT32;
ctype = GAL_TYPE_FLOAT32;
- disp_fmt = GAL_TABLE_DISPLAY_FMT_GENERAL;
+ disp_fmt = GAL_TABLE_DISPLAY_FMT_FLOAT;
disp_width = 8;
disp_precision = 3;
p->upperlimit = 1;
+ oiflag[ OCOL_UPPERLIMIT_SKEW ] = oiflag[ CCOL_UPPERLIMIT_SKEW ] = 1;
break;
case UI_KEY_RIVERAVE:
name = "RIVER_AVE";
- unit = p->input->unit ? p->input->unit : "pixelunit";
+ unit = MKCATALOG_NO_UNIT;
ocomment = NULL;
ccomment = "Average river value surrounding this clump.";
otype = GAL_TYPE_INVALID;
ctype = GAL_TYPE_FLOAT32;
disp_fmt = GAL_TABLE_DISPLAY_FMT_GENERAL;
disp_width = 10;
- disp_precision = 4;
- ciflag[ CCOL_RIV_NUM ] = 1;
- ciflag[ CCOL_RIV_SUM ] = 1;
+ disp_precision = 5;
+ ciflag[ CCOL_RIV_NUM ] = ciflag[ CCOL_RIV_SUM ] = 1;
break;
case UI_KEY_RIVERNUM:
@@ -874,17 +925,16 @@ columns_define_alloc(struct mkcatalogparams *p)
disp_fmt = GAL_TABLE_DISPLAY_FMT_FLOAT;
disp_width = 10;
disp_precision = 3;
- oiflag[ OCOL_SUM ] = 1;
- oiflag[ OCOL_SUM_VAR ] = 1;
- ciflag[ CCOL_SUM ] = 1;
- ciflag[ CCOL_SUM_VAR ] = 1;
- ciflag[ CCOL_RIV_SUM ] = 1;
- ciflag[ CCOL_RIV_SUM_VAR ] = 1;
+ oiflag[ OCOL_SUM ] = ciflag[ CCOL_SUM ] = 1;
+ oiflag[ OCOL_SUM_VAR ] = ciflag[ CCOL_SUM_VAR ] = 1;
+ ciflag[ CCOL_RIV_NUM ] = 1;
+ ciflag[ CCOL_RIV_SUM ] = 1;
+ ciflag[ CCOL_RIV_SUM_VAR ] = 1;
break;
case UI_KEY_SKY:
name = "SKY";
- unit = p->input->unit ? p->input->unit : "pixelunit";
+ unit = MKCATALOG_NO_UNIT;
ocomment = "Average input sky value.";
ccomment = ocomment;
otype = GAL_TYPE_FLOAT32;
@@ -892,15 +942,13 @@ columns_define_alloc(struct mkcatalogparams *p)
disp_fmt = GAL_TABLE_DISPLAY_FMT_GENERAL;
disp_width = 10;
disp_precision = 4;
- oiflag[ OCOL_NUM ] = 1;
- oiflag[ OCOL_SUMSKY ] = 1;
- ciflag[ CCOL_NUM ] = 1;
- ciflag[ CCOL_SUMSKY ] = 1;
+ oiflag[ OCOL_NUM ] = ciflag[ CCOL_NUM ] = 1;
+ oiflag[ OCOL_SUMSKY ] = ciflag[ CCOL_SUMSKY ] = 1;
break;
case UI_KEY_STD:
name = "STD";
- unit = p->input->unit ? p->input->unit : "pixelunit";
+ unit = MKCATALOG_NO_UNIT;
ocomment = "Average of input standard deviation.";
ccomment = ocomment;
otype = GAL_TYPE_FLOAT32;
@@ -908,10 +956,8 @@ columns_define_alloc(struct mkcatalogparams *p)
disp_fmt = GAL_TABLE_DISPLAY_FMT_GENERAL;
disp_width = 10;
disp_precision = 4;
- oiflag[ OCOL_NUM ] = 1;
- oiflag[ OCOL_SUMSTD ] = 1;
- ciflag[ CCOL_NUM ] = 1;
- ciflag[ CCOL_SUMSTD ] = 1;
+ oiflag[ OCOL_NUM ] = ciflag[ CCOL_NUM ] = 1;
+ oiflag[ OCOL_SUMSTD ] = ciflag[ CCOL_SUMSTD ] = 1;
break;
case UI_KEY_SEMIMAJOR:
@@ -924,12 +970,18 @@ columns_define_alloc(struct mkcatalogparams *p)
disp_fmt = GAL_TABLE_DISPLAY_FMT_FLOAT;
disp_width = 10;
disp_precision = 3;
- oiflag[ OCOL_VXX ] = 1;
- oiflag[ OCOL_VYY ] = 1;
- oiflag[ OCOL_VXY ] = 1;
- ciflag[ CCOL_VXX ] = 1;
- ciflag[ CCOL_VYY ] = 1;
- ciflag[ CCOL_VXY ] = 1;
+ oiflag[ OCOL_SUMWHT ] = ciflag[ CCOL_SUMWHT ] = 1;
+ oiflag[ OCOL_VX ] = ciflag[ CCOL_VX ] = 1;
+ oiflag[ OCOL_VY ] = ciflag[ CCOL_VY ] = 1;
+ oiflag[ OCOL_VXX ] = ciflag[ CCOL_VXX ] = 1;
+ oiflag[ OCOL_VYY ] = ciflag[ CCOL_VYY ] = 1;
+ oiflag[ OCOL_VXY ] = ciflag[ CCOL_VXY ] = 1;
+ oiflag[ OCOL_NUMALL ] = ciflag[ CCOL_NUMALL ] = 1;
+ oiflag[ OCOL_GX ] = ciflag[ CCOL_GX ] = 1;
+ oiflag[ OCOL_GY ] = ciflag[ CCOL_GY ] = 1;
+ oiflag[ OCOL_GXX ] = ciflag[ CCOL_GXX ] = 1;
+ oiflag[ OCOL_GYY ] = ciflag[ CCOL_GYY ] = 1;
+ oiflag[ OCOL_GXY ] = ciflag[ CCOL_GXY ] = 1;
break;
case UI_KEY_SEMIMINOR:
@@ -942,12 +994,18 @@ columns_define_alloc(struct mkcatalogparams *p)
disp_fmt = GAL_TABLE_DISPLAY_FMT_FLOAT;
disp_width = 10;
disp_precision = 3;
- oiflag[ OCOL_VXX ] = 1;
- oiflag[ OCOL_VYY ] = 1;
- oiflag[ OCOL_VXY ] = 1;
- ciflag[ CCOL_VXX ] = 1;
- ciflag[ CCOL_VYY ] = 1;
- ciflag[ CCOL_VXY ] = 1;
+ oiflag[ OCOL_SUMWHT ] = ciflag[ CCOL_SUMWHT ] = 1;
+ oiflag[ OCOL_VX ] = ciflag[ CCOL_VX ] = 1;
+ oiflag[ OCOL_VY ] = ciflag[ CCOL_VY ] = 1;
+ oiflag[ OCOL_VXX ] = ciflag[ CCOL_VXX ] = 1;
+ oiflag[ OCOL_VYY ] = ciflag[ CCOL_VYY ] = 1;
+ oiflag[ OCOL_VXY ] = ciflag[ CCOL_VXY ] = 1;
+ oiflag[ OCOL_NUMALL ] = ciflag[ CCOL_NUMALL ] = 1;
+ oiflag[ OCOL_GX ] = ciflag[ CCOL_GX ] = 1;
+ oiflag[ OCOL_GY ] = ciflag[ CCOL_GY ] = 1;
+ oiflag[ OCOL_GXX ] = ciflag[ CCOL_GXX ] = 1;
+ oiflag[ OCOL_GYY ] = ciflag[ CCOL_GYY ] = 1;
+ oiflag[ OCOL_GXY ] = ciflag[ CCOL_GXY ] = 1;
break;
case UI_KEY_AXISRATIO:
@@ -960,12 +1018,18 @@ columns_define_alloc(struct mkcatalogparams *p)
disp_fmt = GAL_TABLE_DISPLAY_FMT_FLOAT;
disp_width = 7;
disp_precision = 3;
- oiflag[ OCOL_VXX ] = 1;
- oiflag[ OCOL_VYY ] = 1;
- oiflag[ OCOL_VXY ] = 1;
- ciflag[ CCOL_VXX ] = 1;
- ciflag[ CCOL_VYY ] = 1;
- ciflag[ CCOL_VXY ] = 1;
+ oiflag[ OCOL_SUMWHT ] = ciflag[ CCOL_SUMWHT ] = 1;
+ oiflag[ OCOL_VX ] = ciflag[ CCOL_VX ] = 1;
+ oiflag[ OCOL_VY ] = ciflag[ CCOL_VY ] = 1;
+ oiflag[ OCOL_VXX ] = ciflag[ CCOL_VXX ] = 1;
+ oiflag[ OCOL_VYY ] = ciflag[ CCOL_VYY ] = 1;
+ oiflag[ OCOL_VXY ] = ciflag[ CCOL_VXY ] = 1;
+ oiflag[ OCOL_NUMALL ] = ciflag[ CCOL_NUMALL ] = 1;
+ oiflag[ OCOL_GX ] = ciflag[ CCOL_GX ] = 1;
+ oiflag[ OCOL_GY ] = ciflag[ CCOL_GY ] = 1;
+ oiflag[ OCOL_GXX ] = ciflag[ CCOL_GXX ] = 1;
+ oiflag[ OCOL_GYY ] = ciflag[ CCOL_GYY ] = 1;
+ oiflag[ OCOL_GXY ] = ciflag[ CCOL_GXY ] = 1;
break;
case UI_KEY_POSITIONANGLE:
@@ -978,12 +1042,18 @@ columns_define_alloc(struct mkcatalogparams *p)
disp_fmt = GAL_TABLE_DISPLAY_FMT_FLOAT;
disp_width = 10;
disp_precision = 3;
- oiflag[ OCOL_VXX ] = 1;
- oiflag[ OCOL_VYY ] = 1;
- oiflag[ OCOL_VXY ] = 1;
- ciflag[ CCOL_VXX ] = 1;
- ciflag[ CCOL_VYY ] = 1;
- ciflag[ CCOL_VXY ] = 1;
+ oiflag[ OCOL_SUMWHT ] = ciflag[ CCOL_SUMWHT ] = 1;
+ oiflag[ OCOL_VX ] = ciflag[ CCOL_VX ] = 1;
+ oiflag[ OCOL_VY ] = ciflag[ CCOL_VY ] = 1;
+ oiflag[ OCOL_VXX ] = ciflag[ CCOL_VXX ] = 1;
+ oiflag[ OCOL_VYY ] = ciflag[ CCOL_VYY ] = 1;
+ oiflag[ OCOL_VXY ] = ciflag[ CCOL_VXY ] = 1;
+ oiflag[ OCOL_NUMALL ] = ciflag[ CCOL_NUMALL ] = 1;
+ oiflag[ OCOL_GX ] = ciflag[ CCOL_GX ] = 1;
+ oiflag[ OCOL_GY ] = ciflag[ CCOL_GY ] = 1;
+ oiflag[ OCOL_GXX ] = ciflag[ CCOL_GXX ] = 1;
+ oiflag[ OCOL_GYY ] = ciflag[ CCOL_GYY ] = 1;
+ oiflag[ OCOL_GXY ] = ciflag[ CCOL_GXY ] = 1;
break;
case UI_KEY_GEOSEMIMAJOR:
@@ -996,12 +1066,12 @@ columns_define_alloc(struct mkcatalogparams *p)
disp_fmt = GAL_TABLE_DISPLAY_FMT_FLOAT;
disp_width = 10;
disp_precision = 3;
- oiflag[ OCOL_GXX ] = 1;
- oiflag[ OCOL_GYY ] = 1;
- oiflag[ OCOL_GXY ] = 1;
- ciflag[ CCOL_GXX ] = 1;
- ciflag[ CCOL_GYY ] = 1;
- ciflag[ CCOL_GXY ] = 1;
+ oiflag[ OCOL_NUMALL ] = ciflag[ CCOL_NUMALL ] = 1;
+ oiflag[ OCOL_GX ] = ciflag[ CCOL_GX ] = 1;
+ oiflag[ OCOL_GY ] = ciflag[ CCOL_GY ] = 1;
+ oiflag[ OCOL_GXX ] = ciflag[ CCOL_GXX ] = 1;
+ oiflag[ OCOL_GYY ] = ciflag[ CCOL_GYY ] = 1;
+ oiflag[ OCOL_GXY ] = ciflag[ CCOL_GXY ] = 1;
break;
case UI_KEY_GEOSEMIMINOR:
@@ -1014,12 +1084,12 @@ columns_define_alloc(struct mkcatalogparams *p)
disp_fmt = GAL_TABLE_DISPLAY_FMT_FLOAT;
disp_width = 10;
disp_precision = 3;
- oiflag[ OCOL_GXX ] = 1;
- oiflag[ OCOL_GYY ] = 1;
- oiflag[ OCOL_GXY ] = 1;
- ciflag[ CCOL_GXX ] = 1;
- ciflag[ CCOL_GYY ] = 1;
- ciflag[ CCOL_GXY ] = 1;
+ oiflag[ OCOL_NUMALL ] = ciflag[ CCOL_NUMALL ] = 1;
+ oiflag[ OCOL_GX ] = ciflag[ CCOL_GX ] = 1;
+ oiflag[ OCOL_GY ] = ciflag[ CCOL_GY ] = 1;
+ oiflag[ OCOL_GXX ] = ciflag[ CCOL_GXX ] = 1;
+ oiflag[ OCOL_GYY ] = ciflag[ CCOL_GYY ] = 1;
+ oiflag[ OCOL_GXY ] = ciflag[ CCOL_GXY ] = 1;
break;
case UI_KEY_GEOAXISRATIO:
@@ -1032,12 +1102,12 @@ columns_define_alloc(struct mkcatalogparams *p)
disp_fmt = GAL_TABLE_DISPLAY_FMT_FLOAT;
disp_width = 7;
disp_precision = 3;
- oiflag[ OCOL_VXX ] = 1;
- oiflag[ OCOL_VYY ] = 1;
- oiflag[ OCOL_VXY ] = 1;
- ciflag[ CCOL_VXX ] = 1;
- ciflag[ CCOL_VYY ] = 1;
- ciflag[ CCOL_VXY ] = 1;
+ oiflag[ OCOL_NUMALL ] = ciflag[ CCOL_NUMALL ] = 1;
+ oiflag[ OCOL_GX ] = ciflag[ CCOL_GX ] = 1;
+ oiflag[ OCOL_GY ] = ciflag[ CCOL_GY ] = 1;
+ oiflag[ OCOL_GXX ] = ciflag[ CCOL_GXX ] = 1;
+ oiflag[ OCOL_GYY ] = ciflag[ CCOL_GYY ] = 1;
+ oiflag[ OCOL_GXY ] = ciflag[ CCOL_GXY ] = 1;
break;
case UI_KEY_GEOPOSITIONANGLE:
@@ -1050,12 +1120,12 @@ columns_define_alloc(struct mkcatalogparams *p)
disp_fmt = GAL_TABLE_DISPLAY_FMT_FLOAT;
disp_width = 10;
disp_precision = 3;
- oiflag[ OCOL_GXX ] = 1;
- oiflag[ OCOL_GYY ] = 1;
- oiflag[ OCOL_GXY ] = 1;
- ciflag[ CCOL_GXX ] = 1;
- ciflag[ CCOL_GYY ] = 1;
- ciflag[ CCOL_GXY ] = 1;
+ oiflag[ OCOL_NUMALL ] = ciflag[ CCOL_NUMALL ] = 1;
+ oiflag[ OCOL_GX ] = ciflag[ CCOL_GX ] = 1;
+ oiflag[ OCOL_GY ] = ciflag[ CCOL_GY ] = 1;
+ oiflag[ OCOL_GXX ] = ciflag[ CCOL_GXX ] = 1;
+ oiflag[ OCOL_GYY ] = ciflag[ CCOL_GYY ] = 1;
+ oiflag[ OCOL_GXY ] = ciflag[ CCOL_GXY ] = 1;
break;
default:
@@ -1085,8 +1155,7 @@ columns_define_alloc(struct mkcatalogparams *p)
allocating the column. */
if(ctype!=GAL_TYPE_INVALID)
{
- /* A clumps image has been given, so allocate space for this
- column. */
+ /* If a clumps labeled image, add this column for the output. */
if(p->clumps)
{
gal_list_data_add_alloc(&p->clumpcols, NULL, ctype, 1,
@@ -1108,16 +1177,18 @@ columns_define_alloc(struct mkcatalogparams *p)
}
- /* If a warning for clumps columns and no clumps image is necessary make
- the warning. */
+ /* If the user has asked for clump-only columns, but no clumps catalog is
+ to be created (the `--clumpscat' option was not given or there were no
+ clumps in the specified image), then print an informative message that
+ the columns in question will be ignored. */
if(noclumpimg)
{
gal_list_str_reverse(&noclumpimg);
- fprintf(stderr, "\n-------\n"
- "WARNING: the following column(s) are unique to "
- "clumps (not objects), but the objects image doesn't have "
- " `WCLUMPS' keyword. So these requested columns will be "
- "ignored.\n\n");
+ fprintf(stderr, "WARNING: the following column(s) are unique to "
+ "clumps (not objects), but the `--clumpscat' option has not "
+ "been called, or there were no clumps in the clumps labeled "
+ "image. Hence, these columns will be ignored in the "
+ "output.\n\n");
for(strtmp=noclumpimg; strtmp!=NULL; strtmp=strtmp->next)
fprintf(stderr, "\t%s\n", strtmp->v);
gal_list_str_free(noclumpimg, 1);
@@ -1201,7 +1272,7 @@ columns_second_order(struct mkcatalog_passparams *pp,
double *row,
int key, int o0c1)
{
double x=NAN, y=NAN, xx=NAN, yy=NAN, xy=NAN;
- double denom, kx=pp->shift[1]+1, ky=pp->shift[0]+1;
+ double denom, kx=pp->shift[1], ky=pp->shift[0];
/* Preparations. */
switch(key)
@@ -1215,15 +1286,15 @@ columns_second_order(struct mkcatalog_passparams *pp,
double *row,
denom = row[ o0c1 ? CCOL_SUMWHT : OCOL_SUMWHT ];
/* First order. */
- x = MKC_RATIO( row[ o0c1 ? CCOL_VX : OCOL_VX ], denom );
- y = MKC_RATIO( row[ o0c1 ? CCOL_VY : OCOL_VY ], denom );
+ x = MKC_RATIO( row[ o0c1 ? CCOL_VX : OCOL_VX ], denom );
+ y = MKC_RATIO( row[ o0c1 ? CCOL_VY : OCOL_VY ], denom );
/* Second order. */
- xx = ( MKC_RATIO( row[ o0c1 ? CCOL_VXX : OCOL_VXX ], denom )
+ xx = ( MKC_RATIO( row[ o0c1 ? CCOL_VXX : OCOL_VXX ], denom )
- (x-kx) * (x-kx) );
- yy = ( MKC_RATIO( row[ o0c1 ? CCOL_VYY : OCOL_VYY ], denom )
+ yy = ( MKC_RATIO( row[ o0c1 ? CCOL_VYY : OCOL_VYY ], denom )
- (y-ky) * (y-ky) );
- xy = ( MKC_RATIO( row[ o0c1 ? CCOL_VXY : OCOL_VXY ], denom )
+ xy = ( MKC_RATIO( row[ o0c1 ? CCOL_VXY : OCOL_VXY ], denom )
- (x-kx) * (y-ky) );
break;
@@ -1236,8 +1307,8 @@ columns_second_order(struct mkcatalog_passparams *pp,
double *row,
denom = row[ o0c1 ? CCOL_NUMALL : OCOL_NUMALL ];
/* First order. */
- x = MKC_RATIO( row[ o0c1 ? CCOL_GX : OCOL_GX ], denom );
- y = MKC_RATIO( row[ o0c1 ? CCOL_GY : OCOL_GY ], denom );
+ x = MKC_RATIO( row[ o0c1 ? CCOL_GX : OCOL_GX ], denom );
+ y = MKC_RATIO( row[ o0c1 ? CCOL_GY : OCOL_GY ], denom );
/* Second order. */
xx = ( MKC_RATIO( row[ o0c1 ? CCOL_GXX : OCOL_GXX ], denom )
@@ -1266,7 +1337,6 @@ columns_second_order(struct mkcatalog_passparams *pp,
double *row,
/* Semi-minor axis. */
case UI_KEY_SEMIMINOR:
case UI_KEY_GEOSEMIMINOR:
- /*printf("\nhere\n");*/
return sqrt( ( xx + yy )/2
- sqrt( (xx - yy)/2 * (xx - yy)/2 + xy * xy ) );
@@ -1391,24 +1461,28 @@ columns_fill(struct mkcatalog_passparams *pp)
break;
case UI_KEY_AREA:
- ((int32_t *)colarr)[oind] = oi[OCOL_NUMALL];
+ ((int32_t *)colarr)[oind] = oi[OCOL_NUM];
break;
case UI_KEY_CLUMPSAREA:
- ((int32_t *)colarr)[oind] = oi[OCOL_C_NUMALL];
+ ((int32_t *)colarr)[oind] = oi[OCOL_C_NUM];
break;
case UI_KEY_WEIGHTAREA:
((int32_t *)colarr)[oind] = oi[OCOL_NUMWHT];
break;
+ case UI_KEY_GEOAREA:
+ ((int32_t *)colarr)[oind] = oi[OCOL_NUMALL];
+ break;
+
case UI_KEY_X:
- ((float *)colarr)[oind] = POS_V_G(oi, OCOL_SUMWHT, OCOL_NUMALL,
+ ((float *)colarr)[oind] = POS_V_G(oi, OCOL_SUMWHT, OCOL_NUMWHT,
OCOL_VX, OCOL_GX);
break;
case UI_KEY_Y:
- ((float *)colarr)[oind] = POS_V_G(oi, OCOL_SUMWHT, OCOL_NUMALL,
+ ((float *)colarr)[oind] = POS_V_G(oi, OCOL_SUMWHT, OCOL_NUMWHT,
OCOL_VY, OCOL_GY);
break;
@@ -1421,12 +1495,12 @@ columns_fill(struct mkcatalog_passparams *pp)
break;
case UI_KEY_CLUMPSX:
- ((float *)colarr)[oind] = POS_V_G(oi, OCOL_C_SUMWHT, OCOL_C_NUMALL,
+ ((float *)colarr)[oind] = POS_V_G(oi, OCOL_C_SUMWHT, OCOL_C_NUMWHT,
OCOL_C_VX, OCOL_C_GX);
break;
case UI_KEY_CLUMPSY:
- ((float *)colarr)[oind] = POS_V_G(oi, OCOL_C_SUMWHT, OCOL_C_NUMALL,
+ ((float *)colarr)[oind] = POS_V_G(oi, OCOL_C_SUMWHT, OCOL_C_NUMWHT,
OCOL_C_VY, OCOL_C_GY);
break;
@@ -1499,7 +1573,9 @@ columns_fill(struct mkcatalog_passparams *pp)
break;
case UI_KEY_MAGNITUDE:
- ((float *)colarr)[oind] = MKC_MAG(oi[ OCOL_SUM ]);
+ ((float *)colarr)[oind] = ( oi[ OCOL_NUM ]>0.0f
+ ? MKC_MAG(oi[ OCOL_SUM ])
+ : NAN );
break;
case UI_KEY_MAGNITUDEERR:
@@ -1614,13 +1690,17 @@ columns_fill(struct mkcatalog_passparams *pp)
break;
case UI_KEY_AREA:
- ((int32_t *)colarr)[cind]=ci[CCOL_NUMALL];
+ ((int32_t *)colarr)[cind]=ci[CCOL_NUM];
break;
case UI_KEY_WEIGHTAREA:
((int32_t *)colarr)[cind]=ci[CCOL_NUMWHT];
break;
+ case UI_KEY_GEOAREA:
+ ((int32_t *)colarr)[cind]=ci[CCOL_NUMALL];
+ break;
+
case UI_KEY_X:
((float *)colarr)[cind] = POS_V_G(ci, CCOL_SUMWHT, CCOL_NUMALL,
CCOL_VX, CCOL_GX);
@@ -1750,7 +1830,7 @@ columns_fill(struct mkcatalog_passparams *pp)
case UI_KEY_AXISRATIO:
((float *)colarr)[cind]
- = ( columns_second_order(pp, ci, UI_KEY_SEMIMINOR, 1)
+ = ( columns_second_order( pp, ci, UI_KEY_SEMIMINOR, 1)
/ columns_second_order(pp, ci, UI_KEY_SEMIMAJOR, 1) );
break;
@@ -1768,7 +1848,7 @@ columns_fill(struct mkcatalog_passparams *pp)
case UI_KEY_GEOAXISRATIO:
((float *)colarr)[cind]
- = ( columns_second_order(pp, ci, UI_KEY_GEOSEMIMINOR, 1)
+ = ( columns_second_order( pp, ci, UI_KEY_GEOSEMIMINOR, 1)
/ columns_second_order(pp, ci, UI_KEY_GEOSEMIMAJOR, 1) );
break;
diff --git a/bin/mkcatalog/main.c b/bin/mkcatalog/main.c
index 1f92630..80b16cb 100644
--- a/bin/mkcatalog/main.c
+++ b/bin/mkcatalog/main.c
@@ -45,7 +45,7 @@ main (int argc, char *argv[])
/* Read the input parameters. */
ui_read_check_inputs_setup(argc, argv, &p);
- /* Run MakeProfiles */
+ /* Run MakeCatalog */
mkcatalog(&p);
/* Free all non-freed allocations. */
diff --git a/bin/mkcatalog/main.h b/bin/mkcatalog/main.h
index 7e63936..ebbe8e9 100644
--- a/bin/mkcatalog/main.h
+++ b/bin/mkcatalog/main.h
@@ -41,6 +41,10 @@ along with Gnuastro. If not, see
<http://www.gnu.org/licenses/>.
#define MKCATALOG_UPPERLIMIT_MINIMUM_NUM 20
+/* Unit string to use if values dataset doesn't have any. */
+#define MKCATALOG_NO_UNIT "input-units"
+
+
/* Intermediate/raw array elements
===============================
@@ -67,14 +71,12 @@ along with Gnuastro. If not, see
<http://www.gnu.org/licenses/>.
enum objectcols
{
OCOL_NUMALL, /* Number of all pixels with this label. */
- OCOL_NUM, /* Number of pixels above threshold. */
+ OCOL_NUM, /* Number of pixels with a value. */
OCOL_SUM, /* Sum of (value-sky) in object. */
OCOL_SUM_VAR, /* Varience of sum (for brightness error). */
OCOL_MEDIAN, /* Median of value in object. */
OCOL_VX, /* Sum of (value-sky) * x. */
OCOL_VY, /* Sum of (value-sky) * y. */
- OCOL_SX, /* Shift along X axis. */
- OCOL_SY, /* Shift along Y axis. */
OCOL_VXX, /* Sum of (value-sky) * x * x. */
OCOL_VYY, /* Sum of (value-sky) * y * y. */
OCOL_VXY, /* Sum of (value-sky) * x * y. */
@@ -106,8 +108,8 @@ enum objectcols
enum clumpcols
{
- CCOL_NUMALL, /* Area of clump irrespective of threshold. */
- CCOL_NUM, /* Area of this clump. */
+ CCOL_NUMALL, /* Number of pixels in clump. */
+ CCOL_NUM, /* Number of values used in clump. */
CCOL_SUM, /* River subtracted brightness. */
CCOL_SUM_VAR, /* Variance of sum (for brightness error). */
CCOL_MEDIAN, /* Median of values in clump. */
@@ -148,9 +150,9 @@ struct mkcatalogparams
/* From command-line */
struct gal_options_common_params cp; /* Common parameters. */
gal_list_i32_t *columnids; /* The desired column codes. */
- char *inputname; /* Input filename. */
- char *objectsfile; /* File name of objects file. */
- char *objectshdu; /* HDU of objects image. */
+ char *objectsfile; /* Input filename. */
+ char *valuesfile; /* File name of objects file. */
+ char *valueshdu; /* HDU of objects image. */
char *clumpsfile; /* File name of objects file. */
char *clumpshdu; /* HDU of objects image. */
char *skyfile; /* File name of sky file. */
@@ -158,33 +160,32 @@ struct mkcatalogparams
char *stdfile; /* File name of sky STD file. */
char *stdhdu; /* HDU of sky STD image. */
+ uint8_t clumpscat; /* ==1: create clumps catalog. */
float zeropoint; /* Zero-point magnitude of object. */
- uint8_t skysubtracted; /* If image is already sky subtracted. */
uint8_t variance; /* Input STD file is actually variance. */
- float threshold; /* Only use values above this threshold.*/
+ uint8_t subtractsky; /* ==1: subtract the Sky from values. */
float sfmagnsigma; /* Surface brightness multiple of sigma.*/
float sfmagarea; /* Surface brightness area (arcsec^2). */
char *upmaskfile; /* Name of upper limit mask file. */
char *upmaskhdu; /* HDU of upper limit mask file. */
size_t upnum; /* Number of upper-limit random samples.*/
- size_t *uprange; /* Range of random positions about target.*/
+ size_t *uprange; /* Range of random pos. around target. */
uint8_t envseed; /* Use the environment for random seed. */
double upsigmaclip[2]; /* Sigma clip to measure upper limit. */
float upnsigma; /* Multiple of sigma to define up-lim. */
+ int32_t checkupperlimit[2]; /* Object & clump ID to check dist. */
/* Internal. */
time_t rawtime; /* Starting time of the program. */
- gal_data_t *input; /* Input. */
+ gal_data_t *values; /* Input. */
gal_data_t *objects; /* Object labels. */
gal_data_t *clumps; /* Clump labels. */
gal_data_t *sky; /* Sky. */
gal_data_t *std; /* Sky standard deviation. */
gal_data_t *upmask; /* Upper limit magnitude mask. */
- float minstd; /* Minimum Standard deviation value. */
float medstd; /* Median standard deviation value. */
float cpscorr; /* Counts-per-second correction. */
- float detsn; /* Minimum detection S/N threshold. */
size_t numobjects; /* Number of object labels in image. */
float clumpsn; /* Clump S/N threshold. */
size_t numclumps; /* Number of clumps in image. */
@@ -193,6 +194,7 @@ struct mkcatalogparams
gal_data_t *tiles; /* Tiles to cover each object. */
char *objectsout; /* Output objects catalog. */
char *clumpsout; /* Output clumps catalog. */
+ char *upcheckout; /* Name of upperlimit check table. */
uint8_t *oiflag; /* Intermediate flags for objects. */
uint8_t *ciflag; /* Intermediate flags for clumps. */
pthread_mutex_t mutex; /* Mutex to change the total numbers. */
@@ -202,6 +204,12 @@ struct mkcatalogparams
const char *rngname; /* Name of random number generator. */
size_t rngmin; /* Minimum possible value of RNG. */
size_t rngdiff; /* Difference of RNG max and min. */
+ uint8_t uprangewarning; /* A warning must be printed. */
+
+ char *usedvaluesfile; /* Ptr to final name used for values. */
+ char *usedclumpsfile; /* Ptr to final name used for clumps. */
+ char *usedskyfile; /* Ptr to final fname used for sky. */
+ char *usedstdfile; /* Ptr to final name used for sky std. */
gal_data_t *wcs_vo; /* Object RA-Dec flux weighted X, Y. */
gal_data_t *wcs_vc; /* Clump RA-Dec flux weighted X, Y. */
diff --git a/bin/mkcatalog/mkcatalog.c b/bin/mkcatalog/mkcatalog.c
index 12b7f13..300dc44 100644
--- a/bin/mkcatalog/mkcatalog.c
+++ b/bin/mkcatalog/mkcatalog.c
@@ -45,6 +45,7 @@ along with Gnuastro. If not, see
<http://www.gnu.org/licenses/>.
#include "mkcatalog.h"
#include "ui.h"
+#include "parse.h"
#include "columns.h"
#include "upperlimit.h"
@@ -52,522 +53,13 @@ along with Gnuastro. If not, see
<http://www.gnu.org/licenses/>.
-/*********************************************************************/
-/************* Definitions and initialization ***************/
-/*********************************************************************/
-/* Both passes are going to need their starting pointers set, so we'll do
- that here. */
-static void
-mkcatalog_initialize_params(struct mkcatalog_passparams *pp)
-{
- struct mkcatalogparams *p=pp->p;
-
- /* Initialize the number of clumps in this object. */
- pp->clumpsinobj=0;
-
-
- /* Initialize the intermediate values. */
- memset(pp->oi, 0, OCOL_NUMCOLS * sizeof *pp->oi);
-
-
- /* Set the shifts in every dimension to avoid round-off errors in large
- numbers for the non-linear calculations. We are using the first pixel
- of each object's tile as the shift parameter to keep the mean
- (average) reasonably near to the standard deviation. Otherwise, when
- the object is far out in the image (large x and y positions), then
- roundoff errors are going to decrease the accuracy of the second order
- calculations. */
- gal_dimension_index_to_coord( ( (float *)(pp->tile->array)
- - (float *)(pp->tile->block->array) ),
- p->input->ndim, p->input->dsize, pp->shift);
-
-
- /* Set the starting and ending indexs of this tile/object. */
- pp->st_i = gal_tile_start_end_ind_inclusive(pp->tile, p->input,
- pp->start_end_inc);
- pp->st_sky = (float *)(p->sky->array) + pp->start_end_inc[0];
- pp->st_std = (float *)(p->std->array) + pp->start_end_inc[0];
- pp->st_o = (int32_t *)(p->objects->array) + pp->start_end_inc[0];
- pp->st_c = ( p->clumps
- ? (int32_t *)(p->clumps->array) + pp->start_end_inc[0]
- : NULL );
-}
-
-
-
-
-
-
-
-
-
-
-
-
-
-
-
-
-
-
-
-
-/*********************************************************************/
-/************* First and second passes ***************/
-/*********************************************************************/
-static void
-mkcatalog_first_pass(struct mkcatalog_passparams *pp)
-{
- struct mkcatalogparams *p=pp->p;
- size_t ndim=p->input->ndim, *dsize=p->input->dsize;
-
- double *oi=pp->oi;
- int32_t *O, *C=NULL;
- size_t d, increment=0, num_increment=1;
- float ss, st, *I, *II, *SK, *ST, *input=p->input->array;
- size_t *c=gal_data_malloc_array(GAL_TYPE_SIZE_T, ndim, __func__, "c");
- size_t *sc=gal_data_malloc_array(GAL_TYPE_SIZE_T, ndim, __func__, "sc");
-
-
- /* Parse each contiguous patch of memory covered by this object. */
- while( pp->start_end_inc[0] + increment <= pp->start_end_inc[1] )
- {
- /* Set the contiguous range to parse, we will check the count
- over the `I' pointer and just increment the rest. */
- O = pp->st_o + increment;
- SK = pp->st_sky + increment;
- ST = pp->st_std + increment;
- if(p->clumps) C = pp->st_c + increment;
- II = ( I = pp->st_i + increment ) + pp->tile->dsize[ndim-1];
-
- /* Parse the tile. */
- do
- {
- /* If this pixel belongs to the requested object then do the
- processing. */
- if( *O==pp->object )
- {
- /* Get the number of clumps in this object: the largest clump
- ID over each object. */
- if( p->clumps && *C>0 )
- pp->clumpsinobj = *C > pp->clumpsinobj ? *C : pp->clumpsinobj;
-
-
- /* Get the coordinates of this point. */
- gal_dimension_index_to_coord(I-input, ndim, dsize, c);
-
-
- /* Calculate the shifted coordinates for second order
- calculations. The coordinate is incremented because from
- now on, the positions are in the FITS standard (starting
- from one).
-
- IMPORTANT NOTE: this is a postfix increment, so after the
- expression (difference) is evaluated, the coordinate is
- going to change. This is necessary because `shift' is also
- starting from zero. */
- for(d=0;d<ndim;++d) sc[d] = c[d]++ - pp->shift[d];
-
-
- /* Do the general geometric (independent of pixel value)
- calculations. */
- oi[ OCOL_NUMALL ]++;
- oi[ OCOL_GX ] += c[1];
- oi[ OCOL_GY ] += c[0];
- oi[ OCOL_GXX ] += sc[1] * sc[1];
- oi[ OCOL_GYY ] += sc[0] * sc[0];
- oi[ OCOL_GXY ] += sc[1] * sc[0];
- if(p->clumps && *C>0)
- {
- oi[ OCOL_C_GX ] += c[1];
- oi[ OCOL_C_GY ] += c[0];
- }
-
-
- /* Start the pixel value related parameters.
-
- ABOUT THE CHECK: The reason this condition is given
- like this is that the `threshold' value is optional
- and we don't want to do multiple checks.
-
- The basic idea is this: when the user doesn't want any
- thresholds applied, then `p->threshold==NAN' and any
- conditional that involves a NaN will fail, so its logical
- negation will be positive and the calculations below will
- be done. However, if the user does specify a threhold and
- the pixel is above the threshold, then (`ss < p->threshold
- * st') will be false and its logical negation will be
- positive, so the pixel will be included. */
- st = p->variance ? sqrt(*ST) : *ST;
- if( !( p->hasblank && isnan(*I) )
- && !( (ss = *I - *SK) < p->threshold * st ) )
- {
- /* General flux summations. */
- oi[ OCOL_NUM ]++;
- oi[ OCOL_SUM ] += ss;
- oi[ OCOL_SUMSKY ] += *SK;
- oi[ OCOL_SUMSTD ] += st;
-
- /* For each pixel, we have a sky contribution to the
- counts and the signal's contribution. The standard
- deviation in the sky is simply `*ST', but the standard
- deviation of the signal (independent of the sky) is
- `sqrt(ss)'. Therefore the total variance of this pixel
- is the variance of the sky added with the absolute
- value of its sky-subtracted flux. We use the absolute
- value, because especially as the signal gets noisy
- there will be negative values, and we don't want them
- to decrease the variance. */
- oi[ OCOL_SUM_VAR ] += (p->variance ? *ST : st*st)+fabs(ss);
-
- /* Get the necessary clump information. */
- if(p->clumps && *C>0)
- {
- oi[ OCOL_C_NUM ]++;
- oi[ OCOL_C_SUM ] += ss;
- }
-
- /* For flux weighted centers, we can only use positive
- values, so do those measurements here. */
- if( ss > 0.0f )
- {
- oi[ OCOL_NUMWHT ]++;
- oi[ OCOL_SUMWHT ] += ss;
- oi[ OCOL_VX ] += ss * c[1];
- oi[ OCOL_VY ] += ss * c[0];
- oi[ OCOL_VXX ] += ss * sc[1] * sc[1];
- oi[ OCOL_VYY ] += ss * sc[0] * sc[0];
- oi[ OCOL_VXY ] += ss * sc[1] * sc[0];
- if(p->clumps && *C>0)
- {
- oi[ OCOL_C_NUMWHT ]++;
- oi[ OCOL_C_SUMWHT ] += ss;
- oi[ OCOL_C_VX ] += ss * c[1];
- oi[ OCOL_C_VY ] += ss * c[0];
- }
- }
- }
- }
-
- /* Increment the other pointers. */
- ++O; ++SK; ++ST; if(p->clumps) ++C;
- }
- while(++I<II);
-
- /* Increment to the next contiguous region of this tile. */
- increment += ( gal_tile_block_increment(p->input, dsize,
- num_increment++, NULL) );
- }
-
- /* Clean up. */
- free(c);
- free(sc);
-}
-
-
-
-
-
-/* Do the second pass */
-static void
-mkcatalog_second_pass(struct mkcatalog_passparams *pp)
-{
- struct mkcatalogparams *p=pp->p;
- size_t ndim=p->input->ndim, *dsize=p->input->dsize;
-
- double *ci, *cir;
- int32_t *O, *C=NULL, nlab, *ngblabs;
- size_t i, ii, d, increment=0, num_increment=1;
- size_t nngb=gal_dimension_num_neighbors(ndim);
- size_t *dinc=gal_dimension_increment(ndim, dsize);
- float ss, st, *I, *II, *SK, *ST, *input=p->input->array;
- int32_t *objects=p->objects->array, *clumps=p->clumps->array;
- size_t *c=gal_data_malloc_array(GAL_TYPE_SIZE_T, ndim, __func__, "c");
- size_t *sc=gal_data_malloc_array(GAL_TYPE_SIZE_T, ndim, __func__, "sc");
-
- /* Allocate array to keep the neighbor labels. */
- ngblabs=gal_data_malloc_array(GAL_TYPE_INT32, nngb, __func__, "ngblabs");
-
- /* Parse each contiguous patch of memory covered by this object. */
- while( pp->start_end_inc[0] + increment <= pp->start_end_inc[1] )
- {
- /* Set the contiguous range to parse, we will check the count
- over the `I' pointer and just increment the rest. */
- O = pp->st_o + increment;
- SK = pp->st_sky + increment;
- ST = pp->st_std + increment;
- if(p->clumps) C = pp->st_c + increment;
- II = ( I = pp->st_i + increment ) + pp->tile->dsize[ndim-1];
-
- /* Parse the next contiguous region of this tile. */
- do
- {
- /* If this pixel belongs to the requested object, is a clumps and
- isn't NAN, then do the processing. `hasblank' is constant, so
- when the input doesn't have any blank values, the `isnan' will
- never be checked. */
- if( *O==pp->object )
- {
- /* We are on a clump. */
- if(p->clumps && *C>0)
- {
- /* Pointer to make things easier. Note that the clump
- labels start from 1, but the array indexs from 0.*/
- ci=&pp->ci[ (*C-1) * CCOL_NUMCOLS ];
-
- /* Get the coordinates of this point. */
- gal_dimension_index_to_coord(I-input, ndim, dsize, c);
-
- /* Shifted coordinates for second order moments, see
- explanations in the first pass.*/
- for(d=0;d<ndim;++d) sc[d] = c[d]++ - pp->shift[d];
-
- /* Geometric measurements (independent of pixel value). */
- ci[ CCOL_NUMALL ]++;
- ci[ CCOL_GX ] += c[1];
- ci[ CCOL_GY ] += c[0];
- ci[ CCOL_GXX ] += sc[1] * sc[1];
- ci[ CCOL_GYY ] += sc[0] * sc[0];
- ci[ CCOL_GXY ] += sc[1] * sc[0];
-
- /* Only use pixels above the threshold, see explanations in
- first pass for an explanation. */
- st = p->variance ? sqrt(*ST) : *ST;
- if( !( p->hasblank && isnan(*I) )
- && !( (ss = *I - *SK) < p->threshold * st ) )
- {
- /* Fill in the necessary information. */
- ci[ CCOL_NUM ]++;
- ci[ CCOL_SUM ] += ss;
- ci[ CCOL_SUMSKY ] += *SK;
- ci[ CCOL_SUMSTD ] += st;
- ci[ CCOL_SUM_VAR ] += ( (p->variance ? *ST : st*st)
- + fabs(ss) );
- if( ss > 0.0f )
- {
- ci[ CCOL_NUMWHT ]++;
- ci[ CCOL_SUMWHT ] += ss;
- ci[ CCOL_VX ] += ss * c[1];
- ci[ CCOL_VY ] += ss * c[0];
- ci[ CCOL_VXX ] += ss * sc[1] * sc[1];
- ci[ CCOL_VYY ] += ss * sc[0] * sc[0];
- ci[ CCOL_VXY ] += ss * sc[1] * sc[0];
- }
- }
- }
-
- /* This pixel is on the diffuse region, check to see if it is
- touching a clump or not, but only if this object actually
- has any clumps. */
- else if(pp->clumpsinobj)
- {
- /* We are on a diffuse (possibly a river) pixel. So the
- value of this pixel has to be added to any of the
- clumps in touches. But since it might touch a labeled
- region more than once, we use `ngblabs' to keep track
- of which label we have already added its value
- to. `ii' is the number of different labels this river
- pixel has already been considered for. `ngblabs' will
- keep the list labels. */
- ii=0;
- memset(ngblabs, 0, nngb*sizeof *ngblabs);
-
- /* Go over the neighbors and see if this pixel is
- touching a clump or not. */
- GAL_DIMENSION_NEIGHBOR_OP(I-input, ndim, dsize, ndim, dinc,
- {
- /* Neighbor's label (mainly for easy reading). */
- nlab=clumps[nind];
-
- /* We only want neighbors that are a clump and part
- of this object and part of the same object. */
- if( nlab>0 && objects[nind]==pp->object)
- {
- /* Go over all already checked labels and make
- sure this clump hasn't already been
- considered. */
- for(i=0;i<ii;++i) if(ngblabs[i]==nlab) break;
-
- /* It hasn't been considered yet: */
- if(i==ii)
- {
- /* Make sure it won't be considered any
- more. */
- ngblabs[ii++] = nlab;
-
- /* To help in reading. */
- cir=&pp->ci[ (nlab-1) * CCOL_NUMCOLS ];
-
- /* Write in the values. */
- cir[ CCOL_RIV_NUM ]++;
- cir[ CCOL_RIV_SUM ] += *I-*SK;
- cir[ CCOL_RIV_SUM_VAR ] +=
- ( (p->variance ? *ST : *ST * *ST)
- + fabs(*I-*SK) );
- }
- }
- });
- }
- }
-
- /* Increment the other pointers. */
- ++O; ++SK; ++ST; if(p->clumps) ++C;
- }
- while(++I<II);
-
- /* Increment to the next contiguous region of this tile. */
- increment += ( gal_tile_block_increment(p->input, dsize,
- num_increment++, NULL) );
- }
-
- /* Clean up. */
- free(c);
- free(sc);
- free(dinc);
- free(ngblabs);
-}
-
-
-
-
-
-static void
-mkcatalog_median_pass(struct mkcatalog_passparams *pp)
-{
- struct mkcatalogparams *p=pp->p;
- size_t ndim=p->input->ndim, *dsize=p->input->dsize;
-
- double *ci;
- gal_data_t *median;
- int32_t *O, *C=NULL;
- gal_data_t **clumpsmed=NULL;
- float ss, st, *I, *II, *SK, *ST;
- size_t i, increment=0, num_increment=1;
- size_t counter=0, *ccounter=NULL, tsize=pp->oi[OCOL_NUM];
- gal_data_t *objmed=gal_data_alloc(NULL, p->input->type, 1, &tsize, NULL, 0,
- p->cp.minmapsize, NULL, NULL, NULL);
-
- /* A small sanity check. */
- if(p->input->type!=GAL_TYPE_FLOAT32)
- error(EXIT_FAILURE, 0, "%s: the input has to be float32 type", __func__);
-
-
- /* Allocate space for the clump medians. */
- if(p->clumps)
- {
- errno=0;
- clumpsmed=malloc(pp->clumpsinobj * sizeof *clumpsmed);
- if(clumpsmed==NULL)
- error(EXIT_FAILURE, errno, "%s: couldn't allocate `clumpsmed' for "
- "%zu clumps", __func__, pp->clumpsinobj);
-
-
- /* Allocate the array necessary to keep the values of each clump. */
- ccounter=gal_data_calloc_array(GAL_TYPE_SIZE_T, pp->clumpsinobj,
- __func__, "ccounter");
- for(i=0;i<pp->clumpsinobj;++i)
- {
- tsize=pp->ci[ i * CCOL_NUMCOLS + CCOL_NUM ];
- clumpsmed[i]=gal_data_alloc(NULL, p->input->type, 1, &tsize, NULL,
- 0, p->cp.minmapsize, NULL, NULL, NULL);
- }
- }
-
-
- /* Parse each contiguous patch of memory covered by this object. */
- while( pp->start_end_inc[0] + increment <= pp->start_end_inc[1] )
- {
- /* Set the contiguous range to parse, we will check the count
- over the `I' pointer and just increment the rest. */
- O = pp->st_o + increment;
- SK = pp->st_sky + increment;
- ST = pp->st_std + increment;
- if(p->clumps) C = pp->st_c + increment;
- II = ( I = pp->st_i + increment ) + pp->tile->dsize[ndim-1];
-
- /* Parse the next contiguous region of this tile. */
- do
- {
- /* If this pixel belongs to the requested object, then do the
- processing. `hasblank' is constant, so when the input doesn't
- have any blank values, the `isnan' will never be checked. */
- st = p->variance ? sqrt(*ST) : *ST;
- if( *O==pp->object
- && !( p->hasblank && isnan(*I) )
- && !( (ss = *I - *SK) < p->threshold * st ))
- {
- /* Copy the value for the whole object. */
- ss = *I - *SK;
- memcpy( gal_data_ptr_increment(objmed->array, counter++,
- p->input->type),
- &ss, gal_type_sizeof(p->input->type) );
-
- /* We are also on a clump. */
- if(p->clumps && *C>0)
- memcpy( gal_data_ptr_increment(clumpsmed[*C-1]->array,
- ccounter[*C-1]++,
- p->input->type),
- &ss, gal_type_sizeof(p->input->type) );
- }
-
- /* Increment the other pointers. */
- ++O; ++SK; ++ST; if(p->clumps) ++C;
- }
- while(++I<II);
-
- /* Increment to the next contiguous region of this tile. */
- increment += ( gal_tile_block_increment(p->input, dsize,
- num_increment++, NULL) );
- }
-
-
- /* Calculate the final medians for objects. */
- median=gal_data_copy_to_new_type_free(gal_statistics_median(objmed, 1),
- GAL_TYPE_FLOAT64);
- pp->oi[OCOL_MEDIAN]=*((double *)(median->array));
- gal_data_free(objmed);
- gal_data_free(median);
-
-
- /* Calculate the median for clumps. */
- if(p->clumps)
- {
- for(i=0;i<pp->clumpsinobj;++i)
- {
- ci=&pp->ci[ i * CCOL_NUMCOLS ];
- median=gal_statistics_median(clumpsmed[i], 1);
- median=gal_data_copy_to_new_type_free(median, GAL_TYPE_FLOAT64);
- ci[ CCOL_MEDIAN ] = ( *((double *)(median->array))
- - (ci[ CCOL_RIV_SUM ]/ci[ CCOL_RIV_NUM ]) );
- gal_data_free(clumpsmed[i]);
- gal_data_free(median);
- }
- free(clumpsmed);
- free(ccounter);
- }
-}
-
-
-
-
-
-
-
-
-
-
-
-
-
-
-
/*********************************************************************/
-/***************** High-level funcitons *******************/
+/************** Manage a single object *******************/
/*********************************************************************/
static void
mkcatalog_clump_starting_index(struct mkcatalog_passparams *pp)
@@ -601,41 +93,56 @@ mkcatalog_single_object(void *in_prm)
{
struct gal_threads_params *tprm=(struct gal_threads_params *)in_prm;
struct mkcatalogparams *p=(struct mkcatalogparams *)(tprm->params);
- size_t ndim=p->input->ndim;
+ size_t ndim=p->objects->ndim;
size_t i;
+ uint8_t *oif=p->oiflag;
struct mkcatalog_passparams pp;
+
/* Initialize the mkcatalog_passparams elements. */
pp.p = p;
pp.clumpstartindex = 0;
pp.rng = p->rng ? gsl_rng_clone(p->rng) : NULL;
- pp.shift = gal_data_malloc_array(GAL_TYPE_SIZE_T, ndim, __func__,
- "pp.shift");
pp.oi = gal_data_malloc_array(GAL_TYPE_FLOAT64, OCOL_NUMCOLS,
__func__, "pp.oi");
+ /* If we have second order measurements, allocate the array keeping the
+ temporary shift values for each object of this thread. Note that the
+ clumps catalog (if requested), will have the same measurements, so its
+ just enough to check the objects. */
+ pp.shift = ( ( oif[ OCOL_GXX ]
+ || oif[ OCOL_GYY ]
+ || oif[ OCOL_GXY ]
+ || oif[ OCOL_VXX ]
+ || oif[ OCOL_VYY ]
+ || oif[ OCOL_VXY ] )
+ ? gal_data_malloc_array(GAL_TYPE_SIZE_T, ndim, __func__,
+ "pp.shift")
+ : NULL );
+
/* If we have upper-limit mode, then allocate the container to keep the
values to calculate the standard deviation. */
- pp.up_vals = p->upperlimit ? gal_data_alloc(NULL, GAL_TYPE_FLOAT32, 1,
- &p->upnum, NULL, 0,
- p->cp.minmapsize, NULL, NULL,
- NULL) : NULL;
+ pp.up_vals = ( p->upperlimit
+ ? gal_data_alloc(NULL, GAL_TYPE_FLOAT32, 1, &p->upnum,
+ NULL, 0, p->cp.minmapsize, NULL, NULL,
+ NULL)
+ : NULL );
/* Fill the desired columns for all the objects given to this thread. */
for(i=0; tprm->indexs[i]!=GAL_BLANK_SIZE_T; ++i)
{
- /* For easy reading, Note that the object IDs start from one while
+ /* For easy reading. Note that the object IDs start from one while
the array positions start from 0. */
pp.ci=NULL;
pp.object = tprm->indexs[i] + 1;
pp.tile = &p->tiles[ tprm->indexs[i] ];
/* Initialize the parameters for this object/tile. */
- mkcatalog_initialize_params(&pp);
+ parse_initialize(&pp);
/* Get the first pass information. */
- mkcatalog_first_pass(&pp);
+ parse_objects(&pp);
/* Currently the second pass is only necessary when there is a clumps
image. */
@@ -652,12 +159,12 @@ mkcatalog_single_object(void *in_prm)
mkcatalog_clump_starting_index(&pp);
/* Get the second pass information. */
- mkcatalog_second_pass(&pp);
+ parse_clumps(&pp);
}
/* If the median is requested, another pass is necessary. */
if( p->oiflag[ OCOL_MEDIAN ] )
- mkcatalog_median_pass(&pp);
+ parse_median(&pp);
/* Calculate the upper limit magnitude (if necessary). */
if(p->upperlimit) upperlimit_calculate(&pp);
@@ -715,29 +222,29 @@ mkcatalog_wcs_conversion(struct mkcatalogparams *p)
/* Flux weighted center positions for clumps and objects. */
if(p->wcs_vo)
{
- gal_wcs_img_to_world(p->wcs_vo, p->input->wcs, 1);
+ gal_wcs_img_to_world(p->wcs_vo, p->objects->wcs, 1);
if(p->wcs_vc)
- gal_wcs_img_to_world(p->wcs_vc, p->input->wcs, 1);
+ gal_wcs_img_to_world(p->wcs_vc, p->objects->wcs, 1);
}
/* Geometric center positions for clumps and objects. */
if(p->wcs_go)
{
- gal_wcs_img_to_world(p->wcs_go, p->input->wcs, 1);
+ gal_wcs_img_to_world(p->wcs_go, p->objects->wcs, 1);
if(p->wcs_gc)
- gal_wcs_img_to_world(p->wcs_gc, p->input->wcs, 1);
+ gal_wcs_img_to_world(p->wcs_gc, p->objects->wcs, 1);
}
/* All clumps flux weighted center. */
if(p->wcs_vcc)
- gal_wcs_img_to_world(p->wcs_vcc, p->input->wcs, 1);
+ gal_wcs_img_to_world(p->wcs_vcc, p->objects->wcs, 1);
/* All clumps geometric center. */
if(p->wcs_gcc)
- gal_wcs_img_to_world(p->wcs_gcc, p->input->wcs, 1);
+ gal_wcs_img_to_world(p->wcs_gcc, p->objects->wcs, 1);
/* Go over all the object columns and fill in the values. */
@@ -796,19 +303,77 @@ mkcatalog_wcs_conversion(struct mkcatalogparams *p)
+void
+mkcatalog_write_inputs_in_comments(struct mkcatalogparams *p,
+ gal_list_str_t **comments, int withsky,
+ int withstd)
+{
+ char *str;
+
+ if(p->cp.tableformat==GAL_TABLE_FORMAT_TXT)
+ {
+ if( asprintf(&str, "--------- Input files ---------")<0 )
+ error(EXIT_FAILURE, 0, "%s: asprintf allocation", __func__);
+ gal_list_str_add(comments, str, 0);
+ }
+
+ if( asprintf(&str, "Objects: %s (hdu: %s).", p->objectsfile, p->cp.hdu)<0 )
+ error(EXIT_FAILURE, 0, "%s: asprintf allocation", __func__);
+ gal_list_str_add(comments, str, 0);
+
+ if(p->clumps)
+ {
+ if(asprintf(&str, "Clumps: %s (hdu: %s).", p->usedclumpsfile,
+ p->clumpshdu)<0)
+ error(EXIT_FAILURE, 0, "%s: asprintf allocation", __func__);
+ gal_list_str_add(comments, str, 0);
+ }
+
+ if(p->values)
+ {
+ if( asprintf(&str, "Values: %s (hdu: %s).", p->usedvaluesfile,
+ p->valueshdu)<0 )
+ error(EXIT_FAILURE, 0, "%s: asprintf allocation", __func__);
+ gal_list_str_add(comments, str, 0);
+ }
+
+ if(withsky && p->sky)
+ {
+ if( asprintf(&str, "Sky: %s (hdu: %s).", p->usedskyfile,
+ p->skyhdu)<0 )
+ error(EXIT_FAILURE, 0, "%s: asprintf allocation", __func__);
+ gal_list_str_add(comments, str, 0);
+ }
+
+ if(withstd && p->std)
+ {
+ if( asprintf(&str, "Sky STD: %s (hdu: %s).", p->usedstdfile,
+ p->stdhdu)<0 )
+ error(EXIT_FAILURE, 0, "%s: asprintf allocation", __func__);
+ gal_list_str_add(comments, str, 0);
+ }
+
+ if(p->upmaskfile)
+ {
+ if( asprintf(&str, "Upperlimit mask: %s (hdu: %s).", p->upmaskfile,
+ p->upmaskhdu)<0 )
+ error(EXIT_FAILURE, 0, "%s: asprintf allocation", __func__);
+ gal_list_str_add(comments, str, 0);
+ }
+}
+
+
+
+
+
/* Write the similar information. */
static gal_list_str_t *
mkcatalog_outputs_same_start(struct mkcatalogparams *p, int o0c1,
char *ObjClump)
{
- float snlim;
char *str, *tstr;
double pixarea=NAN;
gal_list_str_t *comments=NULL;
- char *skyfile=p->skyfile ? p->skyfile : p->inputname;
- char *stdfile=p->stdfile ? p->stdfile : p->inputname;
- char *clumpsfile=p->clumpsfile ? p->clumpsfile : p->inputname;
- char *objectsfile=p->objectsfile ? p->objectsfile : p->inputname;
if( asprintf(&str, "%s catalog of %s", o0c1 ? "Object" : "Clump",
PROGRAM_STRING)<0 )
@@ -832,44 +397,11 @@ mkcatalog_outputs_same_start(struct mkcatalogparams *p,
int o0c1,
gal_list_str_add(&comments, str, 0);
- if(p->cp.tableformat==GAL_TABLE_FORMAT_TXT)
- {
- if( asprintf(&str, "--------- Input files ---------")<0 )
- error(EXIT_FAILURE, 0, "%s: asprintf allocation", __func__);
- gal_list_str_add(&comments, str, 0);
- }
-
- if( asprintf(&str, "Values: %s (hdu: %s).", p->inputname, p->cp.hdu)<0 )
- error(EXIT_FAILURE, 0, "%s: asprintf allocation", __func__);
- gal_list_str_add(&comments, str, 0);
-
- if( asprintf(&str, "Objects: %s (hdu: %s).", objectsfile, p->objectshdu)<0 )
- error(EXIT_FAILURE, 0, "%s: asprintf allocation", __func__);
- gal_list_str_add(&comments, str, 0);
-
- if(p->clumps)
- {
- if(asprintf(&str, "Clumps: %s (hdu: %s).", clumpsfile, p->clumpshdu)<0)
- error(EXIT_FAILURE, 0, "%s: asprintf allocation", __func__);
- gal_list_str_add(&comments, str, 0);
- }
-
- if( asprintf(&str, "Sky: %s (hdu: %s).", skyfile, p->skyhdu)<0 )
- error(EXIT_FAILURE, 0, "%s: asprintf allocation", __func__);
- gal_list_str_add(&comments, str, 0);
+ /* Write the basic information. */
+ mkcatalog_write_inputs_in_comments(p, &comments, 1, 1);
- if( asprintf(&str, "Sky STD: %s (hdu: %s).", stdfile, p->stdhdu)<0 )
- error(EXIT_FAILURE, 0, "%s: asprintf allocation", __func__);
- gal_list_str_add(&comments, str, 0);
-
- if(p->upmaskfile)
- {
- if( asprintf(&str, "Upperlimit mask: %s (hdu: %s).", p->upmaskfile,
- p->upmaskhdu)<0 )
- error(EXIT_FAILURE, 0, "%s: asprintf allocation", __func__);
- gal_list_str_add(&comments, str, 0);
- }
+ /* Write other supplimentary information. */
if(p->cp.tableformat==GAL_TABLE_FORMAT_TXT)
{
if( asprintf(&str, "--------- Supplimentary information ---------")<0 )
@@ -877,9 +409,9 @@ mkcatalog_outputs_same_start(struct mkcatalogparams *p, int
o0c1,
gal_list_str_add(&comments, str, 0);
}
- if(p->input->wcs)
+ if(p->objects->wcs)
{
- pixarea=gal_wcs_pixel_area_arcsec2(p->input->wcs);
+ pixarea=gal_wcs_pixel_area_arcsec2(p->objects->wcs);
if( isnan(pixarea)==0 )
{
if( asprintf(&str, "Pixel area (arcsec^2): %g", pixarea)<0 )
@@ -896,7 +428,7 @@ mkcatalog_outputs_same_start(struct mkcatalogparams *p, int
o0c1,
}
/* Print surface brightness limits. */
- if( !isnan(p->zeropoint) && !isnan(p->sfmagnsigma) )
+ if( !isnan(p->medstd) && !isnan(p->zeropoint) && !isnan(p->sfmagnsigma) )
{
/* Per pixel. */
if( asprintf(&str, "%g sigma surface brightness (magnitude/pixel): "
@@ -950,23 +482,6 @@ mkcatalog_outputs_same_start(struct mkcatalogparams *p,
int o0c1,
gal_list_str_add(&comments, str, 0);
}
- snlim = o0c1 ? p->clumpsn : p->detsn;
- if( !isnan(snlim) )
- {
- if( asprintf(&str, "%s limiting signal-to-noise ratio: %.3f", ObjClump,
- snlim)<0 )
- error(EXIT_FAILURE, 0, "%s: asprintf allocation", __func__);
- gal_list_str_add(&comments, str, 0);
- }
-
- if(o0c1==0)
- {
- if( asprintf(&str, "(NOTE: S/N limit above is for pseudo-detections, "
- "not objects.)")<0 )
- error(EXIT_FAILURE, 0, "%s: asprintf allocation", __func__);
- gal_list_str_add(&comments, str, 0);
- }
-
if(p->cpscorr>1.0f)
{
if( asprintf(&str, "Counts-per-second correction: %.3f", p->cpscorr)<0 )
@@ -974,68 +489,8 @@ mkcatalog_outputs_same_start(struct mkcatalogparams *p,
int o0c1,
gal_list_str_add(&comments, str, 0);
}
- if( !isnan(p->threshold) )
- {
- if( asprintf(&str, "**IMPORTANT** Pixel threshold (multiple of local "
- "std): %.3f", p->threshold)<0 )
- error(EXIT_FAILURE, 0, "%s: asprintf allocation", __func__);
- gal_list_str_add(&comments, str, 0);
- }
-
-
if(p->upperlimit)
- {
- if(p->cp.tableformat==GAL_TABLE_FORMAT_TXT)
- {
- if(asprintf(&str, "--------- Upper-limit measurement ---------")<0)
- error(EXIT_FAILURE, 0, "%s: asprintf allocation", __func__);
- gal_list_str_add(&comments, str, 0);
- }
-
- if( asprintf(&str, "Number of random samples: %zu", p->upnum)<0 )
- error(EXIT_FAILURE, 0, "%s: asprintf allocation", __func__);
- gal_list_str_add(&comments, str, 0);
-
- if(p->uprange)
- {
- if( asprintf(&str, "Range of random samples about target: %zu, %zu",
- p->uprange[1], p->uprange[0])<0 )
- error(EXIT_FAILURE, 0, "%s: asprintf allocation", __func__);
- gal_list_str_add(&comments, str, 0);
- }
-
- if( asprintf(&str, "Random number generator name: %s", p->rngname)<0 )
- error(EXIT_FAILURE, 0, "%s: asprintf allocation", __func__);
- gal_list_str_add(&comments, str, 0);
-
- if( asprintf(&str, "Random number generator seed: %"PRIu64, p->seed)<0 )
- error(EXIT_FAILURE, 0, "%s: asprintf allocation", __func__);
- gal_list_str_add(&comments, str, 0);
-
- if( asprintf(&str, "Multiple of STD used for sigma-clipping: %.3f",
- p->upsigmaclip[0])<0 )
- error(EXIT_FAILURE, 0, "%s: asprintf allocation", __func__);
- gal_list_str_add(&comments, str, 0);
-
- if(p->upsigmaclip[1]>=1.0f)
- {
- if( asprintf(&str, "Number of clips for sigma-clipping: %.0f",
- p->upsigmaclip[1])<0 )
- error(EXIT_FAILURE, 0, "%s: asprintf allocation", __func__);
- }
- else
- {
- if( asprintf(&str, "Tolerance level to sigma-clipping: %.3f",
- p->upsigmaclip[1])<0 )
- error(EXIT_FAILURE, 0, "%s: asprintf allocation", __func__);
- }
- gal_list_str_add(&comments, str, 0);
-
- if( asprintf(&str, "Multiple of sigma-clipped STD for upper-limit: "
- "%.3f", p->upnsigma)<0 )
- error(EXIT_FAILURE, 0, "%s: asprintf allocation", __func__);
- gal_list_str_add(&comments, str, 0);
- }
+ upperlimit_write_comments(p, &comments, 1);
@@ -1099,13 +554,16 @@ mkcatalog_write_outputs(struct mkcatalogparams *p)
}
/* Inform the user. */
- if(p->clumpsout==p->objectsout)
- printf(" - Output catalog: %s\n", p->objectsout);
- else
+ if(!p->cp.quiet)
{
- printf(" - Output objects catalog: %s\n", p->objectsout);
- if(p->clumps)
- printf(" - Output clumps catalog: %s\n", p->clumpsout);
+ if(p->clumpsout==p->objectsout)
+ printf(" - Output catalog: %s\n", p->objectsout);
+ else
+ {
+ printf(" - Output objects catalog: %s\n", p->objectsout);
+ if(p->clumps)
+ printf(" - Output clumps catalog: %s\n", p->clumpsout);
+ }
}
}
@@ -1138,21 +596,17 @@ mkcatalog(struct mkcatalogparams *p)
it to assign a column to the clumps in the final catalog. */
if( p->cp.numthreads > 1 ) pthread_mutex_init(&p->mutex, NULL);
-
/* Do the processing on each thread. */
gal_threads_spin_off(mkcatalog_single_object, p, p->numobjects,
p->cp.numthreads);
-
/* Post-thread processing, for example to convert image coordinates to RA
and Dec. */
mkcatalog_wcs_conversion(p);
-
/* Write the filled columns into the output. */
mkcatalog_write_outputs(p);
-
/* Destroy the mutex. */
if( p->cp.numthreads>1 ) pthread_mutex_destroy(&p->mutex);
}
diff --git a/bin/mkcatalog/mkcatalog.h b/bin/mkcatalog/mkcatalog.h
index 4e93c35..be94551 100644
--- a/bin/mkcatalog/mkcatalog.h
+++ b/bin/mkcatalog/mkcatalog.h
@@ -31,11 +31,11 @@ struct mkcatalog_passparams
int32_t object; /* Object that is currently working on. */
size_t clumpsinobj; /* The number of clumps in this object. */
gal_data_t *tile; /* The tile to pass-over. */
- float *st_i; /* Starting pointer for input image. */
- int32_t *st_o; /* Starting pointer for objects image. */
- int32_t *st_c; /* Starting pointer for clumps image. */
- float *st_sky; /* Starting pointer for input image. */
- float *st_std; /* Starting pointer for input image. */
+ int32_t *st_o; /* Starting pointer for object labels. */
+ int32_t *st_c; /* Starting pointer for clump labels. */
+ float *st_v; /* Starting pointer for values array. */
+ float *st_sky; /* Starting pointer for Sky array. */
+ float *st_std; /* Starting pointer for Sky STD array. */
size_t start_end_inc[2]; /* Starting and ending indexs. */
size_t *shift; /* Shift coordinates. */
gsl_rng *rng; /* Random number generator. */
@@ -44,6 +44,11 @@ struct mkcatalog_passparams
};
void
+mkcatalog_write_inputs_in_comments(struct mkcatalogparams *p,
+ gal_list_str_t **comments, int withsky,
+ int withstd);
+
+void
mkcatalog(struct mkcatalogparams *p);
#endif
diff --git a/bin/mkcatalog/parse.c b/bin/mkcatalog/parse.c
new file mode 100644
index 0000000..e672a59
--- /dev/null
+++ b/bin/mkcatalog/parse.c
@@ -0,0 +1,641 @@
+/*********************************************************************
+MakeCatalog - Make a catalog from an input and labeled image.
+MakeCatalog is part of GNU Astronomy Utilities (Gnuastro) package.
+
+Original author:
+ Mohammad Akhlaghi <address@hidden>
+Contributing author(s):
+Copyright (C) 2018, Free Software Foundation, Inc.
+
+Gnuastro is free software: you can redistribute it and/or modify it
+under the terms of the GNU General Public License as published by the
+Free Software Foundation, either version 3 of the License, or (at your
+option) any later version.
+
+Gnuastro is distributed in the hope that it will be useful, but
+WITHOUT ANY WARRANTY; without even the implied warranty of
+MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the GNU
+General Public License for more details.
+
+You should have received a copy of the GNU General Public License
+along with Gnuastro. If not, see <http://www.gnu.org/licenses/>.
+**********************************************************************/
+#include <config.h>
+
+#include <math.h>
+#include <stdio.h>
+#include <errno.h>
+#include <error.h>
+#include <float.h>
+#include <string.h>
+#include <stdlib.h>
+
+#include <gnuastro/data.h>
+#include <gnuastro/dimension.h>
+#include <gnuastro/statistics.h>
+
+#include "main.h"
+#include "mkcatalog.h"
+
+#include "parse.h"
+
+
+
+
+
+/* Both passes are going to need their starting pointers set, so we'll do
+ that here. */
+void
+parse_initialize(struct mkcatalog_passparams *pp)
+{
+ struct mkcatalogparams *p=pp->p;
+
+ size_t i, ndim=p->objects->ndim;
+ size_t *start_end=pp->start_end_inc;
+
+ /* Initialize the number of clumps in this object. */
+ pp->clumpsinobj=0;
+
+
+ /* Initialize the intermediate values to zero. */
+ memset(pp->oi, 0, OCOL_NUMCOLS * sizeof *pp->oi);
+
+
+ /* Set the shifts in every dimension to avoid round-off errors in large
+ numbers for the non-linear calculations. We are using the first pixel
+ of each object's tile as the shift parameter to keep the mean
+ (average) reasonably near to the standard deviation. Otherwise, when
+ the object is far out in the image (large x and y positions), then
+ roundoff errors are going to decrease the accuracy of the second order
+ calculations. */
+ if(pp->shift)
+ {
+ /* Get the coordinates of the tile's starting point. */
+ gal_dimension_index_to_coord( ( (float *)(pp->tile->array)
+ - (float *)(pp->tile->block->array) ),
+ ndim, p->objects->dsize, pp->shift);
+
+ /* Change their counting to start from 1, not zero, since we will be
+ using them as FITS coordinates. */
+ for(i=0;i<ndim;++i) ++pp->shift[i];
+ }
+
+
+ /* Set the starting and ending indexs of this tile/object on all (the
+ possible) input arrays. */
+ pp->st_o = gal_tile_start_end_ind_inclusive(pp->tile, p->objects,
+ start_end);
+ pp->st_c = (p->clumps
+ ? (int32_t *)(p->clumps->array) + start_end[0] : NULL);
+ pp->st_v = (p->values
+ ? (float *)(p->values->array) + start_end[0] : NULL);
+ pp->st_sky = ( p->sky
+ ? ( p->sky->size==p->objects->size
+ ? (float *)(p->sky->array) + start_end[0]
+ : NULL )
+ : NULL);
+ pp->st_std = ( p->std
+ ? ( p->std->size==p->objects->size
+ ? (float *)(p->std->array) + start_end[0]
+ : NULL )
+ : NULL );
+}
+
+
+
+
+
+void
+parse_objects(struct mkcatalog_passparams *pp)
+{
+ uint8_t *oif=pp->p->oiflag;
+ struct mkcatalogparams *p=pp->p;
+ size_t ndim=p->objects->ndim, *dsize=p->objects->dsize;
+
+ double *oi=pp->oi;
+ size_t d, increment=0, num_increment=1;
+ float st, sval, *V=NULL, *SK=NULL, *ST=NULL;
+ int32_t *O, *OO, *C=NULL, *objects=p->objects->array;
+ float *std=p->std?p->std->array:NULL, *sky=p->sky?p->sky->array:NULL;
+
+ /* If a any tile processing is necessary. */
+ size_t tid = ( ( (p->sky && pp->st_sky == NULL )
+ || ( p->std && pp->st_std == NULL ) )
+ ? 0 : GAL_BLANK_SIZE_T );
+
+ /* Coordinate shift. */
+ size_t *sc = ( pp->shift
+ ? gal_data_malloc_array(GAL_TYPE_SIZE_T, ndim, __func__,
+ "sc")
+ : NULL );
+
+ /* If any coordinate columns are requested. */
+ size_t *c = (
+ /* Coordinate-related columns. */
+ ( oif[ OCOL_GX ]
+ || oif[ OCOL_GY ]
+ || oif[ OCOL_VX ]
+ || oif[ OCOL_VY ]
+ || oif[ OCOL_C_GX ]
+ || oif[ OCOL_C_GY ]
+ || sc
+ /* When the sky and its STD are tiles, we'll also need
+ the coordinate to find which tile a pixel belongs
+ to. */
+ || tid==GAL_BLANK_SIZE_T )
+ ? gal_data_malloc_array(GAL_TYPE_SIZE_T, ndim, __func__, "c")
+ : NULL );
+
+
+ /* Parse each contiguous patch of memory covered by this object. */
+ while( pp->start_end_inc[0] + increment <= pp->start_end_inc[1] )
+ {
+ /* Set the contiguous range to parse. The pixel-to-pixel counting
+ along the fastest dimension will be done over the `O' pointer. */
+ if( p->clumps ) C = pp->st_c + increment;
+ if( p->values ) V = pp->st_v + increment;
+ if( p->sky && pp->st_sky ) SK = pp->st_sky + increment;
+ if( p->std && pp->st_std ) ST = pp->st_std + increment;
+ OO = ( O = pp->st_o + increment ) + pp->tile->dsize[ndim-1];
+
+ /* Parse the tile. */
+ do
+ {
+ /* If this pixel belongs to the requested object then do the
+ processing. */
+ if( *O==pp->object )
+ {
+ /* INTERNAL: Get the number of clumps in this object: it is
+ the largest clump ID over each object. */
+ if( p->clumps && *C>0 )
+ pp->clumpsinobj = *C > pp->clumpsinobj ? *C : pp->clumpsinobj;
+
+
+ /* Add to the area of this object. */
+ if(oif[ OCOL_NUMALL ]) oi[ OCOL_NUMALL ]++;
+
+
+ /* Geometric coordinate measurements. */
+ if(c)
+ {
+ /* Convert the index to coordinate. */
+ gal_dimension_index_to_coord(O-objects, ndim, dsize, c);
+
+ /* If we need tile-ID, get the tile ID now. */
+ if(tid!=GAL_BLANK_SIZE_T)
+ tid=gal_tile_full_id_from_coord(&p->cp.tl, c);
+
+ /* Do the general geometric (independent of pixel value)
+ calculations. */
+ if(oif[ OCOL_GX ]) oi[ OCOL_GX ] += c[1]+1;
+ if(oif[ OCOL_GY ]) oi[ OCOL_GY ] += c[0]+1;
+ if(pp->shift)
+ {
+ /* Calculate the shifted coordinates for second order
+ calculations. The coordinate is incremented because
+ from now on, the positions are in the FITS standard
+ (starting from one). */
+ for(d=0;d<ndim;++d) sc[d] = c[d] + 1 - pp->shift[d];
+
+ /* Include the shifted values, note that the second
+ order moments are never needed independently, they
+ are used together to find the ellipticity
+ parameters. */
+ oi[ OCOL_GXX ] += sc[1] * sc[1];
+ oi[ OCOL_GYY ] += sc[0] * sc[0];
+ oi[ OCOL_GXY ] += sc[1] * sc[0];
+ }
+ if(p->clumps && *C>0)
+ {
+ if(oif[ OCOL_C_NUMALL ]) oi[ OCOL_C_NUMALL ]++;
+ if(oif[ OCOL_C_GX ]) oi[ OCOL_C_GX ] += c[1]+1;
+ if(oif[ OCOL_C_GY ]) oi[ OCOL_C_GY ] += c[0]+1;
+ }
+ }
+
+
+ /* Value related measurements. */
+ if( p->values && !( p->hasblank && isnan(*V) ) )
+ {
+ /* General flux summations. */
+ if(oif[ OCOL_NUM ]) oi[ OCOL_NUM ]++;
+ if(oif[ OCOL_SUM ]) oi[ OCOL_SUM ] += *V;
+
+ /* Get the necessary clump information. */
+ if(p->clumps && *C>0)
+ {
+ if(oif[ OCOL_C_NUM ]) oi[ OCOL_C_NUM ]++;
+ if(oif[ OCOL_C_SUM ]) oi[ OCOL_C_SUM ] += *V;
+ }
+
+ /* For flux weighted centers, we can only use positive
+ values, so do those measurements here. */
+ if( *V > 0.0f )
+ {
+ if(oif[ OCOL_NUMWHT ]) oi[ OCOL_NUMWHT ]++;
+ if(oif[ OCOL_SUMWHT ]) oi[ OCOL_SUMWHT ] += *V;
+ if(oif[ OCOL_VX ]) oi[ OCOL_VX ] += *V*(c[1]+1);
+ if(oif[ OCOL_VY ]) oi[ OCOL_VY ] += *V*(c[0]+1);
+ if(pp->shift)
+ {
+ oi[ OCOL_VXX ] += *V * sc[1] * sc[1];
+ oi[ OCOL_VYY ] += *V * sc[0] * sc[0];
+ oi[ OCOL_VXY ] += *V * sc[1] * sc[0];
+ }
+ if(p->clumps && *C>0)
+ {
+ if(oif[ OCOL_C_NUMWHT ]) oi[ OCOL_C_NUMWHT ]++;
+ if(oif[ OCOL_C_SUMWHT ]) oi[ OCOL_C_SUMWHT ] += *V;
+ if(oif[ OCOL_C_VX ])
+ oi[ OCOL_C_VX ] += *V * (c[1]+1);
+ if(oif[ OCOL_C_VY ])
+ oi[ OCOL_C_VY ] += *V * (c[0]+1);
+ }
+ }
+ }
+
+
+ /* Sky value based measurements. */
+ if(p->sky)
+ if(oif[ OCOL_SUMSKY ])
+ oi[ OCOL_SUMSKY ] += pp->st_sky ? *SK : sky[tid];
+
+
+ /* Sky standard deviation based measurements.*/
+ if(p->std)
+ {
+ sval = pp->st_std ? *ST : std[tid];
+ st = p->variance ? sqrt(sval) : sval;
+ if(oif[ OCOL_SUMSTD ]) oi[ OCOL_SUMSTD ] += st;
+ /* For each pixel, we have a sky contribution to the
+ counts and the signal's contribution. The standard
+ deviation in the sky is simply `sval', but the
+ standard deviation of the signal (independent of the
+ sky) is `sqrt(*V)'. Therefore the total variance of
+ this pixel is the variance of the sky added with the
+ absolute value of its sky-subtracted flux. We use the
+ absolute value, because especially as the signal gets
+ noisy there will be negative values, and we don't want
+ them to decrease the variance. */
+ if(oif[ OCOL_SUM_VAR ])
+ oi[ OCOL_SUM_VAR ] += ( (p->variance ? sval : st*st)
+ + fabs(*V) );
+ }
+ }
+
+ /* Increment the other pointers. */
+ if( p->values ) ++V;
+ if( p->clumps ) ++C;
+ if( p->sky && pp->st_sky ) ++SK;
+ if( p->std && pp->st_std ) ++ST;
+ }
+ while(++O<OO);
+
+ /* Increment to the next contiguous region of this tile. */
+ increment += ( gal_tile_block_increment(p->objects, dsize,
+ num_increment++, NULL) );
+ }
+
+ /* Clean up. */
+ if(c) free(c);
+ if(sc) free(sc);
+}
+
+
+
+
+
+/* Parse over the clumps within an object. */
+void
+parse_clumps(struct mkcatalog_passparams *pp)
+{
+ struct mkcatalogparams *p=pp->p;
+ size_t ndim=p->objects->ndim, *dsize=p->objects->dsize;
+
+ double *ci, *cir;
+ uint8_t *cif=p->ciflag;
+ int32_t *O, *OO, *C=NULL, nlab;
+ float st, sval, *V=NULL, *SK=NULL, *ST=NULL;
+ size_t i, ii, d, increment=0, num_increment=1;
+ size_t nngb=gal_dimension_num_neighbors(ndim);
+ int32_t *objects=p->objects->array, *clumps=p->clumps->array;
+ float *std=p->std?p->std->array:NULL, *sky=p->sky?p->sky->array:NULL;
+
+ /* If a any tile processing is necessary. */
+ size_t tid = ( ( (p->sky && pp->st_sky == NULL )
+ || ( p->std && pp->st_std == NULL ) )
+ ? 0 : GAL_BLANK_SIZE_T );
+
+ /* Coordinate shift. */
+ size_t *sc = ( pp->shift
+ ? gal_data_malloc_array(GAL_TYPE_SIZE_T, ndim, __func__,
+ "sc")
+ : NULL );
+
+ /* If any coordinate columns are requested. */
+ size_t *c = ( ( cif[ CCOL_GX ]
+ || cif[ CCOL_GY ]
+ || cif[ CCOL_VX ]
+ || cif[ CCOL_VY ]
+ || sc
+ || tid==GAL_BLANK_SIZE_T )
+ ? gal_data_malloc_array(GAL_TYPE_SIZE_T, ndim, __func__, "c")
+ : NULL );
+
+ /* Preparations for neighbor parsing. */
+ int32_t *ngblabs=( ( cif[ CCOL_RIV_NUM ]
+ || cif[ CCOL_RIV_SUM ]
+ || cif[ CCOL_RIV_SUM_VAR ] )
+ ? gal_data_malloc_array(GAL_TYPE_INT32, nngb, __func__,
+ "ngblabs")
+ : NULL );
+ size_t *dinc = ngblabs ? gal_dimension_increment(ndim, dsize) : NULL;
+
+
+ /* Parse each contiguous patch of memory covered by this object. */
+ while( pp->start_end_inc[0] + increment <= pp->start_end_inc[1] )
+ {
+ /* Set the contiguous range to parse. The pixel-to-pixel counting
+ along the fastest dimension will be done over the `O' pointer. */
+ C = pp->st_c + increment;
+ if( p->values ) V = pp->st_v + increment;
+ if( p->sky && pp->st_sky ) SK = pp->st_sky + increment;
+ if( p->std && pp->st_std ) ST = pp->st_std + increment;
+ OO = ( O = pp->st_o + increment ) + pp->tile->dsize[ndim-1];
+
+ /* Parse the tile */
+ do
+ {
+ /* If this pixel belongs to the requested object then do the
+ processing. */
+ if( *O==pp->object )
+ {
+ /* We are on a clump. */
+ if(p->clumps && *C>0)
+ {
+ /* Pointer to make things easier. Note that the clump
+ labels start from 1, but the array indexs from 0.*/
+ ci=&pp->ci[ (*C-1) * CCOL_NUMCOLS ];
+
+ /* Add to the area of this object. */
+ if(cif[ CCOL_NUMALL ]) ci[ CCOL_NUMALL ]++;
+
+ /* Raw-position related measurements. */
+ if(c)
+ {
+ /* Get "C" the coordinates of this point. */
+ gal_dimension_index_to_coord(O-objects, ndim, dsize, c);
+
+ /* If we need tile-ID, get the tile ID now. */
+ if(tid!=GAL_BLANK_SIZE_T)
+ tid=gal_tile_full_id_from_coord(&p->cp.tl, c);
+
+ /* General geometric (independent of pixel value)
+ calculations. */
+ if(cif[ CCOL_GX ]) ci[ CCOL_GX ] += c[1]+1;
+ if(cif[ CCOL_GY ]) ci[ CCOL_GY ] += c[0]+1;
+ if(pp->shift)
+ {
+ /* Shifted coordinates for second order moments,
+ see explanations in the first pass.*/
+ for(d=0;d<ndim;++d) sc[d] = c[d] + 1 - pp->shift[d];
+
+ /* Raw second-order measurements. */
+ ci[ CCOL_GXX ] += sc[1] * sc[1];
+ ci[ CCOL_GYY ] += sc[0] * sc[0];
+ ci[ CCOL_GXY ] += sc[1] * sc[0];
+ }
+ }
+
+ /* Value related measurements, see `parse_objects' for
+ comments. */
+ if( p->values && !( p->hasblank && isnan(*V) ) )
+ {
+ /* Fill in the necessary information. */
+ if(cif[ CCOL_NUM ]) ci[ CCOL_NUM ]++;
+ if(cif[ CCOL_SUM ]) ci[ CCOL_SUM ] += *V;
+ if( *V > 0.0f )
+ {
+ if(cif[ CCOL_NUMWHT ]) ci[ CCOL_NUMWHT ]++;
+ if(cif[ CCOL_SUMWHT ]) ci[ CCOL_SUMWHT ] += *V;
+ if(cif[ CCOL_VX ]) ci[ CCOL_VX ]+=*V*(c[1]+1);
+ if(cif[ CCOL_VY ]) ci[ CCOL_VY ]+=*V*(c[0]+1);
+ if(pp->shift)
+ {
+ ci[ CCOL_VXX ] += *V * sc[1] * sc[1];
+ ci[ CCOL_VYY ] += *V * sc[0] * sc[0];
+ ci[ CCOL_VXY ] += *V * sc[1] * sc[0];
+ }
+ }
+ }
+
+ /* Sky based measurements. */
+ if(p->sky)
+ if(cif[ CCOL_SUMSKY ])
+ ci[ CCOL_SUMSKY ] += pp->st_sky ? *SK : sky[tid];
+
+ /* Sky Standard deviation based measurements, see
+ `parse_objects' for comments. */
+ if(p->std)
+ {
+ sval = pp->st_std ? *ST : std[tid];
+ st = p->variance ? sqrt(sval) : sval;
+ if(cif[ CCOL_SUMSTD ]) ci[ CCOL_SUMSTD ] += st;
+ if(cif[ CCOL_SUM_VAR ])
+ ci[ CCOL_SUM_VAR ] += ( (p->variance ? sval : st*st)
+ + fabs(*V) );
+ }
+ }
+
+ /* This pixel is on the diffuse region (and the object
+ actually has clumps). If any river-based measurements are
+ necessary check to see if it is touching a clump or not,
+ but only if this object actually has any clumps. */
+ else if(ngblabs && pp->clumpsinobj)
+ {
+ /* We are on a diffuse (possibly a river) pixel. So the
+ value of this pixel has to be added to any of the
+ clumps in touches. But since it might touch a labeled
+ region more than once, we use `ngblabs' to keep track
+ of which label we have already added its value
+ to. `ii' is the number of different labels this river
+ pixel has already been considered for. `ngblabs' will
+ keep the list labels. */
+ ii=0;
+ memset(ngblabs, 0, nngb*sizeof *ngblabs);
+
+ /* Go over the neighbors and see if this pixel is
+ touching a clump or not. */
+ GAL_DIMENSION_NEIGHBOR_OP(O-objects, ndim, dsize, ndim,
+ dinc,
+ {
+ /* Neighbor's label (mainly for easy reading). */
+ nlab=clumps[nind];
+
+ /* We only want neighbors that are a clump and part
+ of this object and part of the same object. */
+ if( nlab>0 && objects[nind]==pp->object)
+ {
+ /* Go over all already checked labels and make
+ sure this clump hasn't already been
+ considered. */
+ for(i=0;i<ii;++i) if(ngblabs[i]==nlab) break;
+
+ /* It hasn't been considered yet: */
+ if(i==ii)
+ {
+ /* Make sure it won't be considered any
+ more. */
+ ngblabs[ii++] = nlab;
+
+ /* To help in reading. */
+ cir=&pp->ci[ (nlab-1) * CCOL_NUMCOLS ];
+
+ /* Write in the necessary values. */
+ if(cif[ CCOL_RIV_NUM ])
+ cir[ CCOL_RIV_NUM ]++;
+
+ if(cif[ CCOL_RIV_SUM ])
+ cir[ CCOL_RIV_SUM ] += *V;
+
+ if(cif[ CCOL_RIV_SUM_VAR ])
+ {
+ sval = pp->st_std ? *ST : std[tid];
+ cir[ CCOL_RIV_SUM_VAR ] += fabs(*V)
+ + (p->variance ? sval : sval*sval);
+ }
+ }
+ }
+ });
+ }
+ }
+
+ /* Increment the other pointers. */
+ ++C;
+ if( p->values ) ++V;
+ if( p->sky && pp->st_sky ) ++SK;
+ if( p->std && pp->st_std ) ++ST;
+ }
+ while(++O<OO);
+
+ /* Increment to the next contiguous region of this tile. */
+ increment += ( gal_tile_block_increment(p->objects, dsize,
+ num_increment++, NULL) );
+ }
+
+ /* Clean up. */
+ if(c) free(c);
+ if(sc) free(sc);
+ if(dinc) free(dinc);
+ if(ngblabs) free(ngblabs);
+}
+
+
+
+
+
+void
+parse_median(struct mkcatalog_passparams *pp)
+{
+ struct mkcatalogparams *p=pp->p;
+ size_t ndim=p->objects->ndim, *dsize=p->objects->dsize;
+
+ float *V;
+ double *ci;
+ gal_data_t *median;
+ int32_t *O, *OO, *C=NULL;
+ gal_data_t **clumpsmed=NULL;
+ size_t i, increment=0, num_increment=1;
+ size_t counter=0, *ccounter=NULL, tsize=pp->oi[OCOL_NUM];
+ gal_data_t *objmed=gal_data_alloc(NULL, p->values->type, 1, &tsize, NULL, 0,
+ p->cp.minmapsize, NULL, NULL, NULL);
+
+ /* Allocate space for the clump medians. */
+ if(p->clumps)
+ {
+ errno=0;
+ clumpsmed=malloc(pp->clumpsinobj * sizeof *clumpsmed);
+ if(clumpsmed==NULL)
+ error(EXIT_FAILURE, errno, "%s: couldn't allocate `clumpsmed' for "
+ "%zu clumps", __func__, pp->clumpsinobj);
+
+
+ /* Allocate the array necessary to keep the values of each clump. */
+ ccounter=gal_data_calloc_array(GAL_TYPE_SIZE_T, pp->clumpsinobj,
+ __func__, "ccounter");
+ for(i=0;i<pp->clumpsinobj;++i)
+ {
+ tsize=pp->ci[ i * CCOL_NUMCOLS + CCOL_NUM ];
+ clumpsmed[i]=gal_data_alloc(NULL, p->values->type, 1, &tsize, NULL,
+ 0, p->cp.minmapsize, NULL, NULL, NULL);
+ }
+ }
+
+
+ /* Parse each contiguous patch of memory covered by this object. */
+ while( pp->start_end_inc[0] + increment <= pp->start_end_inc[1] )
+ {
+ /* Set the contiguous range to parse. The pixel-to-pixel counting
+ along the fastest dimension will be done over the `O' pointer. */
+ V = pp->st_v + increment;
+ if(p->clumps) C = pp->st_c + increment;
+ OO = ( O = pp->st_o + increment ) + pp->tile->dsize[ndim-1];
+
+ /* Parse the next contiguous region of this tile. */
+ do
+ {
+ /* If this pixel belongs to the requested object, then do the
+ processing. `hasblank' is constant, so when the values doesn't
+ have any blank values, the `isnan' will never be checked. */
+ if( *O==pp->object && !( p->hasblank && isnan(*V) ) )
+ {
+ /* Copy the value for the whole object. */
+ memcpy( gal_data_ptr_increment(objmed->array, counter++,
+ p->values->type), V,
+ gal_type_sizeof(p->values->type) );
+
+ /* We are also on a clump. */
+ if(p->clumps && *C>0)
+ memcpy( gal_data_ptr_increment(clumpsmed[*C-1]->array,
+ ccounter[*C-1]++,
+ p->values->type), V,
+ gal_type_sizeof(p->values->type) );
+ }
+
+ /* Increment the other pointers. */
+ ++V;
+ if(p->clumps) ++C;
+ }
+ while(++O<OO);
+
+ /* Increment to the next contiguous region of this tile. */
+ increment += ( gal_tile_block_increment(p->objects, dsize,
+ num_increment++, NULL) );
+ }
+
+
+ /* Calculate the final medians for objects. */
+ median=gal_data_copy_to_new_type_free(gal_statistics_median(objmed, 1),
+ GAL_TYPE_FLOAT64);
+ pp->oi[OCOL_MEDIAN]=*((double *)(median->array));
+ gal_data_free(objmed);
+ gal_data_free(median);
+
+
+ /* Calculate the median for clumps. */
+ if(p->clumps)
+ {
+ for(i=0;i<pp->clumpsinobj;++i)
+ {
+ ci=&pp->ci[ i * CCOL_NUMCOLS ];
+ median=gal_statistics_median(clumpsmed[i], 1);
+ median=gal_data_copy_to_new_type_free(median, GAL_TYPE_FLOAT64);
+ ci[ CCOL_MEDIAN ] = ( *((double *)(median->array))
+ - (ci[ CCOL_RIV_SUM ]/ci[ CCOL_RIV_NUM ]) );
+ gal_data_free(clumpsmed[i]);
+ gal_data_free(median);
+ }
+ free(clumpsmed);
+ free(ccounter);
+ }
+}
diff --git a/bin/mkcatalog/upperlimit.h b/bin/mkcatalog/parse.h
similarity index 76%
copy from bin/mkcatalog/upperlimit.h
copy to bin/mkcatalog/parse.h
index 143252f..5dc0238 100644
--- a/bin/mkcatalog/upperlimit.h
+++ b/bin/mkcatalog/parse.h
@@ -5,7 +5,7 @@ MakeCatalog is part of GNU Astronomy Utilities (Gnuastro)
package.
Original author:
Mohammad Akhlaghi <address@hidden>
Contributing author(s):
-Copyright (C) 2015-2018, Free Software Foundation, Inc.
+Copyright (C) 2018, Free Software Foundation, Inc.
Gnuastro is free software: you can redistribute it and/or modify it
under the terms of the GNU General Public License as published by the
@@ -20,10 +20,19 @@ General Public License for more details.
You should have received a copy of the GNU General Public License
along with Gnuastro. If not, see <http://www.gnu.org/licenses/>.
**********************************************************************/
-#ifndef UPPERLIMIT_H
-#define UPPERLIMIT_H
+#ifndef PARSE_H
+#define PARSE_H
void
-upperlimit_calculate(struct mkcatalog_passparams *pp);
+parse_initialize(struct mkcatalog_passparams *pp);
+
+void
+parse_objects(struct mkcatalog_passparams *pp);
+
+void
+parse_clumps(struct mkcatalog_passparams *pp);
+
+void
+parse_median(struct mkcatalog_passparams *pp);
#endif
diff --git a/bin/mkcatalog/ui.c b/bin/mkcatalog/ui.c
index ce329c8..8f02eb4 100644
--- a/bin/mkcatalog/ui.c
+++ b/bin/mkcatalog/ui.c
@@ -119,14 +119,15 @@ ui_initialize_options(struct mkcatalogparams *p,
cp->coptions = gal_commonopts_options;
/* Specific to this program. */
+ p->medstd = NAN;
p->sfmagnsigma = NAN;
p->sfmagarea = NAN;
p->upnsigma = NAN;
p->zeropoint = NAN;
- p->threshold = NAN;
p->upsigmaclip[0] = NAN;
p->upsigmaclip[1] = NAN;
-
+ p->checkupperlimit[0] = GAL_BLANK_INT32;
+ p->checkupperlimit[1] = GAL_BLANK_INT32;
/* Modify common options. */
for(i=0; !gal_options_is_last(&cp->coptions[i]); ++i)
@@ -138,6 +139,9 @@ ui_initialize_options(struct mkcatalogparams *p,
case GAL_OPTIONS_KEY_TYPE:
case GAL_OPTIONS_KEY_SEARCHIN:
case GAL_OPTIONS_KEY_IGNORECASE:
+ case GAL_OPTIONS_KEY_WORKOVERCH:
+ case GAL_OPTIONS_KEY_INTERPNUMNGB:
+ case GAL_OPTIONS_KEY_INTERPONLYBLANK:
cp->coptions[i].flags=OPTION_HIDDEN;
cp->coptions[i].mandatory=GAL_OPTIONS_NOT_MANDATORY;
break;
@@ -146,15 +150,6 @@ ui_initialize_options(struct mkcatalogparams *p,
cp->coptions[i].mandatory=GAL_OPTIONS_MANDATORY;
break;
}
-
- /* Select by group. */
- switch(cp->coptions[i].group)
- {
- case GAL_OPTIONS_GROUP_TESSELLATION:
- cp->coptions[i].doc=NULL; /* Necessary to remove title. */
- cp->coptions[i].flags=OPTION_HIDDEN;
- break;
- }
}
}
@@ -189,10 +184,10 @@ parse_opt(int key, char *arg, struct argp_state *state)
/* Read the non-option tokens (arguments): */
case ARGP_KEY_ARG:
- if(p->inputname)
+ if(p->objectsfile)
argp_error(state, "only one argument (input file) should be given");
else
- p->inputname=arg;
+ p->objectsfile=arg;
break;
@@ -253,6 +248,91 @@ ui_column_codes_ll(struct argp_option *option, char *arg,
+/* Prepare the upper-limit distribution parameters. */
+void *
+ui_check_upperlimit(struct argp_option *option, char *arg,
+ char *filename, size_t lineno, void *params)
+{
+ size_t i;
+ char *str;
+ double *d;
+ gal_data_t *raw;
+ struct mkcatalogparams *p=(struct mkcatalogparams *)params;
+
+ /* Write. */
+ if(lineno==-1)
+ {
+ if(p->checkupperlimit[1]==GAL_BLANK_INT32)
+ {
+ if( asprintf(&str, "%d", p->checkupperlimit[0])<0 )
+ error(EXIT_FAILURE, 0, "%s: asprintf allocation", __func__);
+ }
+ else
+ if( asprintf(&str, "%d,%d", p->checkupperlimit[0],
+ p->checkupperlimit[1])<0 )
+ error(EXIT_FAILURE, 0, "%s: asprintf allocation", __func__);
+ return str;
+ }
+
+ /* Read */
+ else
+ {
+ /* If the option is already set, just return. */
+ if(option->set) return NULL;
+
+ /* Read the list of numbers as an array. */
+ raw=gal_options_parse_list_of_numbers(arg, filename, lineno);
+
+ /* Make sure there is at most only two numbers given. */
+ if(raw->size>2)
+ error_at_line(EXIT_FAILURE, 0, filename, lineno, "`%s' (value to "
+ "`--%s') contains %zu numbers, but only one or two "
+ "are acceptable.\n\n"
+ "With this option MakeCatalog will write all the "
+ "positions and values of the random distribution for "
+ "one particular labeled region into a table. The "
+ "given value(s) is(are) the label identifier.\n\n"
+ "With one value the distribution for an object will "
+ "be printed: the givne number will be interpretted as "
+ "the requested object's label. With two values, the "
+ "distribution for a specific clump will be written. "
+ "The first will be interpretted as the clump's host "
+ "object label and the second as the clump's label "
+ "within the object", arg, option->name, raw->size);
+
+ /* Make sure the given values are integers and that they are larger
+ than zero. */
+ d=raw->array;
+ for(i=0;i<raw->size;++i)
+ {
+ if( ceil(d[i]) != d[i])
+ error_at_line(EXIT_FAILURE, 0, filename, lineno, "%g (value "
+ "number %zu given to `--%s') is not an "
+ "integer. The value(s) to this option are "
+ "object/clump labels/identifiers, so they must be "
+ "integers", d[i], i+1, option->name);
+ if( d[i]<=0 )
+ error_at_line(EXIT_FAILURE, 0, filename, lineno, "%g (value "
+ "number %zu given to `--%s') is not positive. "
+ "The value(s) to this option are object/clump "
+ "labels/identifiers, so they must be positive "
+ "integers", d[i], i+1, option->name);
+ }
+
+ /* Write the values in. */
+ p->checkupperlimit[0] = d[0];
+ p->checkupperlimit[1] = raw->size==2 ? d[1] : GAL_BLANK_INT32;
+
+ /* For no un-used variable warning. This function doesn't need the
+ pointer.*/
+ return NULL;
+ }
+}
+
+
+
+
+
@@ -272,12 +352,22 @@ ui_column_codes_ll(struct argp_option *option, char *arg,
/*************** Sanity Check *******************/
/**************************************************************/
/* Read and check ONLY the options. When arguments are involved, do the
- check in `ui_check_options_and_arguments'.
+ check in `ui_check_options_and_arguments'. */
static void
ui_read_check_only_options(struct mkcatalogparams *p)
{
+ /* If an upper-limit check table is requested with a specific clump, but
+ no clump catalog has been requested, then abort and inform the
+ user. */
+ if( p->checkupperlimit[1]!=GAL_BLANK_INT32 && p->clumpscat==0 )
+ error(EXIT_FAILURE, 0, "no clumps catalog is requested, hence "
+ "`--checkupperlimit' is only available for objects (one value "
+ "must be given to it).\n\n"
+ "To ask for a clumps catalog, please append `--clumpscat' to the "
+ "command calling MakeCatalog.\n\n"
+ "If you want the upperlimit check table for an object, only give "
+ "one value (the object's label) to `--checkupperlimit'.");
}
-*/
@@ -285,11 +375,11 @@ ui_read_check_only_options(struct mkcatalogparams *p)
static void
ui_check_options_and_arguments(struct mkcatalogparams *p)
{
- /* Make sure an input file name was given and if it was a FITS file, that
- a HDU is also given. */
- if(p->inputname)
+ /* Make sure the main input file name (for the object labels) was given
+ and if it was a FITS file, that a HDU is also given. */
+ if(p->objectsfile)
{
- if( gal_fits_name_is_fits(p->inputname) && p->cp.hdu==NULL )
+ if( gal_fits_name_is_fits(p->objectsfile) && p->cp.hdu==NULL )
error(EXIT_FAILURE, 0, "no HDU specified. When the input is a FITS "
"file, a HDU must also be specified, you can use the `--hdu' "
"(`-h') option and give it the HDU number (starting from "
@@ -322,6 +412,48 @@ ui_check_options_and_arguments(struct mkcatalogparams *p)
/**************************************************************/
/*************** Preparations *******************/
/**************************************************************/
+/* If the user hasn't explicitly specified a filename for input,
+ MakeCatalog will use other given file names. */
+static void
+ui_set_filenames(struct mkcatalogparams *p)
+{
+ p->usedclumpsfile = p->clumpsfile ? p->clumpsfile : p->objectsfile;
+
+ p->usedvaluesfile = p->valuesfile ? p->valuesfile : p->objectsfile;
+
+ p->usedskyfile = ( p->skyfile
+ ? p->skyfile
+ : ( p->valuesfile ? p->valuesfile : p->objectsfile ) );
+
+ p->usedstdfile = ( p->stdfile
+ ? p->stdfile
+ : ( p->valuesfile ? p->valuesfile : p->objectsfile ) );
+}
+
+
+
+
+
+/* The clumps and objects images must be integer type, so we'll use this
+ function to avoid having to write the error message two times. */
+static void
+ui_check_type_int(char *filename, char *hdu, uint8_t type)
+{
+ if( type==GAL_TYPE_FLOAT32 || type==GAL_TYPE_FLOAT64 )
+ error(EXIT_FAILURE, 0, "%s (hdu: %s): type %s not acceptable as "
+ "labels input. labeled images must have an integer datatype.\n\n"
+ "If you are sure the extension contains only integer values but "
+ "it is just stored in a floating point container, you can "
+ "put it in an integer container with Gnuastro's Arithmetic "
+ "program using this command:\n\n"
+ " $ astarithmetic %s int32 -h%s", filename, hdu,
+ gal_type_name(type, 1), filename, hdu);
+}
+
+
+
+
+
/* If a WCS structure is present, then read its basic information to use in
the table meta-data. */
static void
@@ -330,24 +462,28 @@ ui_wcs_info(struct mkcatalogparams *p)
char *c;
size_t i;
+ /* Read the WCS meta-data. */
+ p->objects->wcs=gal_wcs_read(p->objectsfile, p->cp.hdu, 0, 0,
+ &p->objects->nwcs);
+
/* Read the basic WCS information. */
- if(p->input->wcs)
+ if(p->objects->wcs)
{
/* Allocate space for the array of strings. */
errno=0;
- p->ctype=malloc(p->input->ndim * sizeof *p->ctype);
+ p->ctype=malloc(p->objects->ndim * sizeof *p->ctype);
if(p->ctype==NULL)
error(EXIT_FAILURE, 0, "%s: %zu bytes for `p->ctype'", __func__,
- p->input->ndim * sizeof *p->ctype);
+ p->objects->ndim * sizeof *p->ctype);
/* Fill in the values. */
- for(i=0;i<p->input->ndim;++i)
+ for(i=0;i<p->objects->ndim;++i)
{
/* CTYPE might contain `-' characters, we just want the first
non-dash characters. The loop will either stop either at the end
or where there is a dash. So we can just replace it with an
end-of-string character. */
- gal_checkset_allocate_copy(p->input->wcs->ctype[i], &p->ctype[i]);
+ gal_checkset_allocate_copy(p->objects->wcs->ctype[i], &p->ctype[i]);
c=p->ctype[i]; while(*c!='\0' && *c!='-') ++c;
*c='\0';
}
@@ -358,353 +494,568 @@ ui_wcs_info(struct mkcatalogparams *p)
+/* The only mandatory input is the objects image, so first read that and
+ make sure its type is correct. */
static void
-ui_preparations_read_inputs(struct mkcatalogparams *p)
+ui_read_labels(struct mkcatalogparams *p)
{
- size_t one=1;
- int readclumps=0;
- char *namestypes, **strarr=NULL;
- gal_data_t *zero, *key=gal_data_array_calloc(1);
- char *skyfile=p->skyfile ? p->skyfile : p->inputname;
- char *stdfile=p->stdfile ? p->stdfile : p->inputname;
- char *clumpsfile=p->clumpsfile ? p->clumpsfile : p->inputname;
- char *objectsfile=p->objectsfile ? p->objectsfile : p->inputname;
+ char *msg;
+ gal_data_t *tmp, *keys=gal_data_array_calloc(2);
+ /* Read it into memory. */
+ p->objects = gal_fits_img_read(p->objectsfile, p->cp.hdu,
+ p->cp.minmapsize);
- /* Read the input image. */
- p->input=gal_fits_img_read_to_type(p->inputname, p->cp.hdu,
- GAL_TYPE_FLOAT32, p->cp.minmapsize);
- p->input->wcs=gal_wcs_read(p->inputname, p->cp.hdu, 0, 0, &p->input->nwcs);
+ /* Make sure it has an integer type. */
+ ui_check_type_int(p->objectsfile, p->cp.hdu, p->objects->type);
- /* Read basic WCS information for final table meta-data. */
- ui_wcs_info(p);
+
+ /* Conver it to `int32' type (if it already isn't). */
+ gal_data_copy_to_new_type_free(p->objects, GAL_TYPE_INT32);
/* Currently MakeCatalog is only implemented for 2D images. */
- if(p->input->ndim!=2)
+ if(p->objects->ndim!=2)
error(EXIT_FAILURE, 0, "%s (hdu %s) has %zu dimensions, MakeCatalog "
- "currently only supports 2D inputs", p->inputname, p->cp.hdu,
- p->input->ndim);
-
+ "currently only supports 2D inputs", p->objectsfile, p->cp.hdu,
+ p->objects->ndim);
- /* See if the input has blank pixels and set the flags appropriately. */
- p->hasblank = gal_blank_present(p->input, 1);
-
- /* Read the object label image and check its size. */
- p->objects = gal_fits_img_read(objectsfile, p->objectshdu,
- p->cp.minmapsize);
- if( gal_data_dsize_is_different(p->input, p->objects) )
- error(EXIT_FAILURE, 0, "`%s' (hdu: %s) and `%s' (hdu: %s) have a"
- "different dimension/size", objectsfile, p->objectshdu,
- p->inputname, p->cp.hdu);
-
-
- /* Read the Sky image and check its size. */
- p->sky=gal_fits_img_read_to_type(skyfile, p->skyhdu, GAL_TYPE_FLOAT32,
- p->cp.minmapsize);
- if( gal_data_dsize_is_different(p->input, p->sky) )
- error(EXIT_FAILURE, 0, "`%s' (hdu: %s) and `%s' (hdu: %s) have a"
- "different dimension/size", skyfile, p->skyhdu, p->inputname,
- p->cp.hdu);
+ /* See if the total number of objects is given in the header keywords. */
+ keys[0].name="NUMLABS";
+ keys[0].type=GAL_TYPE_SIZE_T;
+ keys[0].array=&p->numobjects;
+ gal_fits_key_read(p->objectsfile, p->cp.hdu, keys, 0, 0);
+ if(keys[0].status) /* status!=0: the key couldn't be read by CFITSIO. */
+ {
+ tmp=gal_statistics_maximum(p->objects);
+ p->numobjects=*((int32_t *)(tmp->array)); /*numobjects is in int32_t.*/
+ gal_data_free(tmp);
+ }
- /* Read the Sky standard deviation image and check its size. */
- p->std=gal_fits_img_read_to_type(stdfile, p->stdhdu, GAL_TYPE_FLOAT32,
- p->cp.minmapsize);
- if( gal_data_dsize_is_different(p->input, p->std) )
- error(EXIT_FAILURE, 0, "`%s' (hdu: %s) and `%s' (hdu: %s) have a"
- "different dimension/size", stdfile, p->stdhdu, p->inputname,
- p->cp.hdu);
+ /* If there were no objects in the input, then inform the user with an
+ error (it is pointless to build a catalog). */
+ if(p->numobjects==0)
+ error(EXIT_FAILURE, 0, "no object labels (non-zero pixels) in "
+ "%s (hdu %s). To make a catalog, labeled regions must be defined",
+ p->objectsfile, p->cp.hdu);
- /* If an upper-limit mask image was given, read it. */
- if(p->upmaskfile)
- {
- /* Read the mask image. */
- p->upmask = gal_fits_img_read(p->upmaskfile, p->upmaskhdu,
- p->cp.minmapsize);
- if( gal_data_dsize_is_different(p->input, p->upmask) )
- error(EXIT_FAILURE, 0, "`%s' (hdu: %s) and `%s' (hdu: %s) have a"
- "different dimension/size", p->upmaskfile, p->upmaskhdu,
- p->inputname, p->cp.hdu);
-
- /* If it isn't an integer type, report an error, otherwise, convert
- it to a uint8_t: with a 1 for all non-zero pixels and 0 for zero
- pixels. */
- zero=gal_data_alloc(NULL, GAL_TYPE_UINT8, 1, &one, NULL, 1, -1,
- NULL, NULL, NULL);
- p->upmask=gal_arithmetic(GAL_ARITHMETIC_OP_NE,
- ( GAL_ARITHMETIC_INPLACE | GAL_ARITHMETIC_FREE
- | GAL_ARITHMETIC_NUMOK ), p->upmask, zero);
- }
+ /* See if the labels image has blank pixels and set the flags
+ appropriately. */
+ p->hasblank = gal_blank_present(p->objects, 1);
- /* Check to see if the objects extension has a `WCLUMPS' keyword and that
- its value is 'yes', `y', or `1'. */
- key->name="WCLUMPS";
- key->type=GAL_TYPE_STRING;
- gal_fits_key_read(objectsfile, p->objectshdu, key, 0, 0);
- if(key->status==KEY_NO_EXIST) readclumps=0;
- else
- {
- if(key->status)
- gal_fits_io_error(key->status, "CFITSIO error while reading "
- "WCLUMPS keyword");
- else
- {
- strarr=key->array;
- if( !strcasecmp(strarr[0], "yes") || !strcasecmp(strarr[0], "y")
- || !strcmp(strarr[0], "1") ) readclumps=1;
- }
- }
+ /* Prepare WCS information for final table meta-data. */
+ ui_wcs_info(p);
/* Read the clumps array if necessary. */
- if(readclumps)
+ if(p->clumpscat)
{
- /* Make sure the user did indeed give the clumps HDU. */
+ /* Make sure the HDU is also given. */
if(p->clumpshdu==NULL)
- error(EXIT_FAILURE, 0, "no `--clumpshdu' given! The WCLUMPS keyword "
- "in %s (hdu: %s) has a value of `%s', so MakeCatalog expects "
- "a clump image.\n\nYou can use the optional `--clumpsfile' "
- "option to give the filename and the mandatory `--clumpshdu' "
- "to specify the extension. If `--clumpsfile' is not given, "
- "MakeCatalog will look into the input file for the given "
- "extension. Alternatively, you can modify/remove this "
- "keyword using Gnuastro's Fits program, please run `$ info "
- "astfits' for more information (press `SPACE' to go down and "
- "`q' to return to the command-line).", objectsfile,
- p->objectshdu, strarr[0]);
-
- /* Read the clumps image and check its size. */
- p->clumps = gal_fits_img_read(clumpsfile, p->clumpshdu,
+ error(EXIT_FAILURE, 0, "%s: no HDU/extension provided for the "
+ "CLUMPS dataset. Please use the `--clumpshdu' option to "
+ "give a specific HDU using its number (counting from zero) "
+ "or name. If the dataset is in another file, please use "
+ "`--clumpsfile' to give the filename. If you don't want any "
+ "clumps catalog output, remove the `--clumpscat' option from "
+ "the command-line or give it a value of zero in a "
+ "configuration file", p->usedclumpsfile);
+
+ /* Read the clumps image. */
+ p->clumps = gal_fits_img_read(p->usedclumpsfile, p->clumpshdu,
p->cp.minmapsize);
- if( gal_data_dsize_is_different(p->input, p->std) )
+
+ /* Check its size. */
+ if( gal_data_dsize_is_different(p->objects, p->clumps) )
error(EXIT_FAILURE, 0, "`%s' (hdu: %s) and `%s' (hdu: %s) have a"
- "different dimension/size", clumpsfile, p->clumpshdu,
- p->inputname, p->cp.hdu);
- }
+ "different dimension/size", p->usedclumpsfile, p->clumpshdu,
+ p->objectsfile, p->cp.hdu);
+ /* Check its type. */
+ ui_check_type_int(p->usedclumpsfile, p->clumpshdu, p->clumps->type);
+ p->clumps=gal_data_copy_to_new_type_free(p->clumps, GAL_TYPE_INT32);
- /* Make sure the object and clump label images have an integer type, then
- to be safe that they have the correct integer type, run
- `gal_data_copy_to_new_type' on them. */
- if( p->objects->type==GAL_TYPE_FLOAT32
- || p->objects->type==GAL_TYPE_FLOAT64
- || (p->clumps && ( p->clumps->type==GAL_TYPE_FLOAT32
- || p->clumps->type==GAL_TYPE_FLOAT64 ) ) )
- {
- if(p->clumps)
+ /* See if there are keywords to help in finding the number. */
+ keys[0].next=&keys[1];
+ keys[0].status=keys[1].status=0;
+ keys[0].name="CLUMPSN"; keys[1].name="NUMLABS";
+ keys[0].type=GAL_TYPE_FLOAT32; keys[1].type=GAL_TYPE_SIZE_T;
+ keys[0].array=&p->clumpsn; keys[1].array=&p->numclumps;
+ gal_fits_key_read(p->usedclumpsfile, p->clumpshdu, keys, 0, 0);
+ if(keys[0].status) p->clumpsn=NAN;
+ if(keys[1].status)
{
- if( asprintf(&namestypes, "However, `%s' (hdu: %s) and `%s' "
- "(hdu: %s) have types of `%s' and `%s' respectively",
- objectsfile, p->objectshdu, clumpsfile, p->clumpshdu,
- gal_type_name(p->objects->type, 1),
- gal_type_name(p->clumps->type, 1) )<0 )
+ if( asprintf(&msg, "%s (hdu: %s): couldn't find/read `NUMLABS' in "
+ "the header keywords, see CFITSIO error above. The "
+ "clumps image must have the total number of clumps "
+ "(irrespective of how many objects there are in the "
+ "image) in this header keyword", p->usedclumpsfile,
+ p->clumpshdu)<0 )
error(EXIT_FAILURE, 0, "%s: asprintf allocation", __func__);
+ gal_fits_io_error(keys[1].status, msg);
}
- else
+
+ /* If there were no clumps, then free the clumps array and set it to
+ NULL, so for the rest of the processing, MakeCatalog things that
+ no clumps image was given. */
+ if(p->numclumps==0)
{
- if( asprintf(&namestypes, "However, %s (hdu: %s) has a type of %s",
- objectsfile, p->objectshdu,
- gal_type_name(p->objects->type, 1))<0 )
- error(EXIT_FAILURE, 0, "%s: asprintf allocation", __func__);
+ /* Just as a sanity check, see if there are any clumps (positive
+ valued pixels) in the array. If there are, then `NUMCLUMPS'
+ wasn't set properly and we should abort with an error. */
+ tmp=gal_statistics_maximum(p->clumps);
+ if( *((int32_t *)(p->clumps->array))>0 )
+ error(EXIT_FAILURE, 0, "%s (hdu: %s): the `NUMCLUMPS' header "
+ "keyword has a value of zero, but there are positive "
+ "pixels in the array, showing that there are clumps in "
+ "image. This is a wrong usage of the `NUMCLUMPS' keyword."
+ "It must contain the total number of clumps (irrespective "
+ "of how many objects there are). Please correct this issue "
+ "and run MakeCatalog again", p->usedclumpsfile,
+ p->clumpshdu);
+
+ /* Since there are no clumps, we won't bother creating a clumps
+ catalog and from this step onward, we'll act as if no clumps
+ catalog was requested. In order to not confuse the user in the
+ end, we'll print a warning first. */
+ fprintf(stderr, "WARNING: %s (hdu %s): there are no clumps "
+ "in the image, therefore no clumps catalog will be "
+ "created.\n", p->usedclumpsfile, p->clumpshdu);
+ gal_data_free(p->clumps);
+ p->clumps=NULL;
}
- error(EXIT_FAILURE, 0, "labeled images (for objects or clumps) must "
- "have an integer datatype. %s.\n\n"
- "If you are sure the images contain only integer values but "
- "are just stored in a floating point container, you can "
- "put them in an integer container with Gnuastro's Arithmetic "
- "program using a command like below:\n\n"
- " $ astarithmetic img.fits int32", namestypes);
}
- p->objects=gal_data_copy_to_new_type_free(p->objects, GAL_TYPE_INT32);
- if(p->clumps)
- p->clumps=gal_data_copy_to_new_type_free(p->clumps, GAL_TYPE_INT32);
/* Clean up. */
- key->name=NULL;
- gal_data_array_free(key, 1, 1);
+ keys[0].name=keys[1].name=NULL;
+ keys[0].array=keys[1].array=NULL;
+ gal_data_array_free(keys, 2, 1);
}
-/* The input images can have extensions to speed up the processing. */
+/* See which inputs are necessary. Ultimate, there are only three extra
+ inputs: a values image, a sky image and a sky standard deviation
+ image. However, there are many raw column measurements. So to keep
+ things clean, we'll just put a value of `1' in the three `values', `sky'
+ and `std' pointers everytime a necessary input is found. */
static void
-ui_preparations_read_keywords(struct mkcatalogparams *p)
+ui_necessary_inputs(struct mkcatalogparams *p, int *values, int *sky,
+ int *std)
{
- char *msg;
- gal_data_t *tmp;
- gal_data_t *keys=gal_data_array_calloc(2);
- char *stdfile=p->stdfile ? p->stdfile : p->inputname;
- char *clumpsfile=p->clumpsfile ? p->clumpsfile : p->inputname;
- char *objectsfile=p->objectsfile ? p->objectsfile : p->inputname;
+ size_t i;
+
+ if(p->upperlimit) *values=1;
+
+ /* Go over all the object columns. Note that the objects and clumps (if
+ the `--clumpcat' option is given) inputs are mandatory and it is not
+ necessary to specify it here. */
+ for(i=0; i<OCOL_NUMCOLS; ++i)
+ if(p->oiflag[i])
+ switch(i)
+ {
+ case OCOL_NUMALL: /* Only object labels. */ break;
+ case OCOL_NUM: *values = 1; break;
+ case OCOL_SUM: *values = 1; break;
+ case OCOL_SUM_VAR: *values = *std = 1; break;
+ case OCOL_MEDIAN: *values = 1; break;
+ case OCOL_VX: *values = 1; break;
+ case OCOL_VY: *values = 1; break;
+ case OCOL_VXX: *values = 1; break;
+ case OCOL_VYY: *values = 1; break;
+ case OCOL_VXY: *values = 1; break;
+ case OCOL_SUMSKY: *sky = 1; break;
+ case OCOL_SUMSTD: *std = 1; break;
+ case OCOL_SUMWHT: *values = 1; break;
+ case OCOL_NUMWHT: *values = 1; break;
+ case OCOL_GX: /* Only object labels. */ break;
+ case OCOL_GY: /* Only object labels. */ break;
+ case OCOL_GXX: /* Only object labels. */ break;
+ case OCOL_GYY: /* Only object labels. */ break;
+ case OCOL_GXY: /* Only object labels. */ break;
+ case OCOL_UPPERLIMIT_B: *values = 1; break;
+ case OCOL_UPPERLIMIT_S: *values = 1; break;
+ case OCOL_UPPERLIMIT_Q: *values = 1; break;
+ case OCOL_UPPERLIMIT_SKEW: *values = 1; break;
+ case OCOL_C_NUMALL: /* Only clump labels. */ break;
+ case OCOL_C_NUM: *values = 1; break;
+ case OCOL_C_SUM: *values = 1; break;
+ case OCOL_C_VX: *values = 1; break;
+ case OCOL_C_VY: *values = 1; break;
+ case OCOL_C_GX: /* Only clump labels. */ break;
+ case OCOL_C_GY: /* Only clump labels. */ break;
+ case OCOL_C_SUMWHT: *values = 1; break;
+ case OCOL_C_NUMWHT: *values = 1; break;
+ default:
+ error(EXIT_FAILURE, 0, "%s: a bug! Please contact us at %s to "
+ "fix the problem. The code %zu is not a recognized "
+ "intermediate OBJECT columns", __func__, PACKAGE_BUGREPORT,
+ i);
+ }
+
+ /* Check the clump elements also. */
+ if(p->clumps)
+ for(i=0; i<CCOL_NUMCOLS; ++i)
+ if(p->ciflag[i])
+ switch(i)
+ {
+ case CCOL_NUMALL: /* Only clump labels. */ break;
+ case CCOL_NUM: *values = 1; break;
+ case CCOL_SUM: *values = 1; break;
+ case CCOL_SUM_VAR: *values = *std = 1; break;
+ case CCOL_MEDIAN: *values = 1; break;
+ case CCOL_RIV_NUM: /* Only clump labels. */ break;
+ case CCOL_RIV_SUM: *values = 1; break;
+ case CCOL_RIV_SUM_VAR: *values = *std = 1; break;
+ case CCOL_VX: *values = 1; break;
+ case CCOL_VY: *values = 1; break;
+ case CCOL_VXX: *values = 1; break;
+ case CCOL_VYY: *values = 1; break;
+ case CCOL_VXY: *values = 1; break;
+ case CCOL_SUMSKY: *sky = 1; break;
+ case CCOL_SUMSTD: *std = 1; break;
+ case CCOL_SUMWHT: *values = 1; break;
+ case CCOL_NUMWHT: *values = 1; break;
+ case CCOL_GX: /* Only clump labels. */ break;
+ case CCOL_GY: /* Only clump labels. */ break;
+ case CCOL_GXX: /* Only clump labels. */ break;
+ case CCOL_GYY: /* Only clump labels. */ break;
+ case CCOL_GXY: /* Only clump labels. */ break;
+ case CCOL_UPPERLIMIT_B: *values = 1; break;
+ case CCOL_UPPERLIMIT_S: *values = 1; break;
+ case CCOL_UPPERLIMIT_Q: *values = 1; break;
+ case CCOL_UPPERLIMIT_SKEW: *values = 1; break;
+ default:
+ error(EXIT_FAILURE, 0, "%s: a bug! Please contact us at %s to "
+ "fix the problem. The code %zu is not a recognized "
+ "intermediate CLUMP column", __func__, PACKAGE_BUGREPORT,
+ i);
+ }
+}
- /* Read the keywords from the standard deviation image. */
- keys[0].next=&keys[1];
- keys[0].name="MINSTD"; keys[1].name="MEDSTD";
- keys[0].type=GAL_TYPE_FLOAT32; keys[1].type=GAL_TYPE_FLOAT32;
- keys[0].array=&p->minstd; keys[1].array=&p->medstd;
- gal_fits_key_read(stdfile, p->stdhdu, keys, 0, 0);
- /* If the two keywords couldn't be read, calculate them. */
- if(keys[0].status)
- {
- /* Calculate the minimum STD. */
- tmp=gal_statistics_minimum(p->std);
- p->minstd=*((float *)(tmp->array));
- gal_data_free(tmp);
- }
- if(keys[1].status)
- {
- /* Calculate the median STD. */
- tmp=gal_statistics_median(p->std, 0);
- p->medstd=*((float *)(tmp->array));
- gal_data_free(tmp);
- /* Alert the user if it wasn't calculated from a header keyword. */
- fprintf(stderr, "---------------\n"
- "Warning: Could not find the `MEDSTD' keyword in `%s' "
- "(hdu: %s). The median standard deviation is thus found on "
- "the (interpolated) standard deviation image. NoiseChisel "
- "finds the median before interpolation which is more "
- "accurate. Ho the reported value in the final catalog will "
- "not be accurate: it will depend on how many tiles were "
- "blank and their spatial position relative to the non-blank "
- "ones.\n"
- "---------------\n", stdfile, p->stdhdu);
- }
- p->cpscorr = p->minstd>1 ? 1.0f : p->minstd;
+/* When the Sky and its standard deviation are given as tiles, we need to
+ define a tile structure. */
+static void
+ui_preparation_check_size_read_tiles(struct mkcatalogparams *p,
+ gal_data_t *in, char *filename,
+ char *hdu)
+{
+ struct gal_tile_two_layer_params *tl=&p->cp.tl;
- /* Read the keywords from the objects image. */
- keys[0].name="DETSN"; keys[1].name="NUMLABS";
- keys[0].type=GAL_TYPE_FLOAT32; keys[1].type=GAL_TYPE_SIZE_T;
- keys[0].array=&p->detsn; keys[1].array=&p->numobjects;
- gal_fits_key_read(objectsfile, p->objectshdu, keys, 0, 0);
- if(keys[0].status) p->detsn=NAN; /* When `status!=0', then */
- if(keys[1].status) /* the key couldn't be read. */
+ /* See if we should treat this dataset as tile values or not. */
+ if( gal_data_dsize_is_different(p->objects, in) )
{
- tmp=gal_statistics_maximum(p->objects);
- p->numobjects=*((int32_t *)(tmp->array)); /*numobjects is in int32_t.*/
- gal_data_free(tmp);
+ /* The `tl' structure is initialized here. But this function may be
+ called multiple times. So, first check if the `tl' structure has
+ already been initialized and if so, don't repeat it. */
+ if(tl->ndim==0)
+ {
+ gal_tile_full_sanity_check(p->objectsfile, p->cp.hdu, p->objects,
+ tl);
+ gal_tile_full_two_layers(p->objects, tl);
+ gal_tile_full_permutation(tl);
+ }
+
+ /* See if the size of the `in' dataset corresponds to the
+ tessellation. */
+ if(in->size!=tl->tottiles)
+ error(EXIT_FAILURE, 0, "%s (hdu: %s): doesn't have the right "
+ "size (%zu elements or pixels).\n\n"
+ "It must either be the same size as `%s' (hdu: `%s'), or "
+ "it must have the same number of elements as the total "
+ "number of tiles in the tessellation (%zu). In the latter "
+ "case, each pixel is assumed to be a fixed value for a "
+ "complete tile.\n\n"
+ "Run with `-P' to see the (tessellation) options/settings "
+ "and their values). For more information on tessellation in "
+ "Gnuastro, please run the following command (use the arrow "
+ "keys for up and down and press `q' to return to the "
+ "command-line):\n\n"
+ " $ info gnuastro tessellation",
+ filename, hdu, in->size, p->objectsfile, p->cp.hdu,
+ tl->tottiles);
}
+}
- /* If there were no objects in the input, then inform the user with an
- error (no catalog was built). */
- if(p->numobjects==0)
- error(EXIT_FAILURE, 0, "no object labels (non-zero pixels) in "
- "%s (hdu %s). To make a catalog, labeled regions must be defined",
- objectsfile, p->objectshdu);
- /* Read the keywords from the clumps image if necessary. */
- if(p->clumps)
+
+/* Subtract `sky' from the input dataset depending on its size (it may be
+ the whole array or a tile-values array).. */
+static void
+ui_subtract_sky(gal_data_t *in, gal_data_t *sky,
+ struct gal_tile_two_layer_params *tl)
+{
+ size_t tid;
+ gal_data_t *tile;
+ float *f, *s, *sf, *skyarr=sky->array;
+
+ /* It is the same size as the input. */
+ if( gal_data_dsize_is_different(in, sky)==0 )
{
- keys[0].name="CLUMPSN"; keys[1].name="NUMLABS";
- keys[0].type=GAL_TYPE_FLOAT32; keys[1].type=GAL_TYPE_SIZE_T;
- keys[0].array=&p->clumpsn; keys[1].array=&p->numclumps;
- gal_fits_key_read(clumpsfile, p->clumpshdu, keys, 0, 0);
- if(keys[0].status) p->clumpsn=NAN;
- if(keys[1].status)
+ f=in->array;
+ sf=(s=sky->array)+sky->size;
+ do *f++-=*s; while(++s<sf);
+ }
+
+ /* It is the same size as the number of tiles. */
+ else if( tl->tottiles==sky->size )
+ {
+ /* Go over all the tiles. */
+ for(tid=0; tid<tl->tottiles; ++tid)
{
- if( asprintf(&msg, "couldn't find/read NUMLABS in the header of "
- "%s (hdu: %s), see error above", clumpsfile,
- p->clumpshdu)<0 )
- error(EXIT_FAILURE, 0, "%s: asprintf allocation", __func__);
- gal_fits_io_error(keys[1].status, msg);
+ /* For easy reading. */
+ tile=&tl->tiles[tid];
+
+ /* Subtract the Sky value from the input image. */
+ GAL_TILE_PO_OISET(int32_t, float, tile, in, 1, 1,
+ {*o-=skyarr[tid];});
}
}
+ /* The size must have been checked before, so if control reaches here, we
+ have a bug! */
+ else
+ error(EXIT_FAILURE, 0, "%s: a bug! Please contact us at %s to fix "
+ "the problem. For some reason, the size doesn't match", __func__,
+ PACKAGE_BUGREPORT);
+}
+
+
+
+
- /* If there were no clumps, then free the clumps array and set it to
- NULL, so for the rest of the processing, MakeCatalog things that no
- clumps image was given. */
- if(p->numclumps==0)
+static void
+ui_preparations_read_inputs(struct mkcatalogparams *p)
+{
+ size_t one=1;
+ gal_data_t *zero;
+ gal_data_t *column;
+ int need_values=0, need_sky=0, need_std=0;
+
+ /* See which inputs are necessary. */
+ ui_necessary_inputs(p, &need_values, &need_sky, &need_std);
+
+
+ /* If the values dataset is necessary, read it in and set the units of
+ the columns from it (if it has any). */
+ if(need_values)
{
- gal_data_free(p->clumps);
- p->clumps=NULL;
+ /* Make sure the HDU is also given. */
+ if(p->valueshdu==NULL)
+ error(EXIT_FAILURE, 0, "%s: no HDU/extension provided for the "
+ "VALUES dataset. Atleast one column needs this dataset. "
+ "Please use the `--valueshdu' option to give a specific HDU "
+ "using its number (counting from zero) or name. If the "
+ "dataset is in another file, please use `--valuesfile' to "
+ "give the filename", p->usedvaluesfile);
+
+ /* Read the values dataset. */
+ p->values=gal_fits_img_read_to_type(p->usedvaluesfile, p->valueshdu,
+ GAL_TYPE_FLOAT32, p->cp.minmapsize);
+
+ /* Make sure it has the correct size. */
+ if( gal_data_dsize_is_different(p->objects, p->values) )
+ error(EXIT_FAILURE, 0, "`%s' (hdu: %s) and `%s' (hdu: %s) have a"
+ "different dimension/size", p->usedvaluesfile, p->valueshdu,
+ p->objectsfile, p->cp.hdu);
+
+ /* Initially, `p->hasblank' was set based on the objects image, but
+ it may happen that the objects image only has zero values for
+ blank pixels, so we'll also do a check on the input image. */
+ p->hasblank = gal_blank_present(p->objects, 1);
+
+ /* Reset the units of the value-based columns if the input dataset
+ has defined units. */
+ if(p->values->unit)
+ {
+ for(column=p->objectcols; column!=NULL; column=column->next)
+ if( !strcmp(column->unit, MKCATALOG_NO_UNIT) )
+ { free(column->unit); column->unit=p->values->unit; }
+ for(column=p->clumpcols; column!=NULL; column=column->next)
+ if( !strcmp(column->unit, MKCATALOG_NO_UNIT) )
+ { free(column->unit); column->unit=p->values->unit; }
+ }
}
- /* Clean up. */
- keys[0].name=keys[1].name=NULL;
- keys[0].array=keys[1].array=NULL;
- gal_data_array_free(keys, 2, 1);
-}
+ /* Read the Sky image and check its size. */
+ if(p->subtractsky || need_sky)
+ {
+ /* Make sure the HDU is also given. */
+ if(p->skyhdu==NULL)
+ error(EXIT_FAILURE, 0, "%s: no HDU/extension provided for the "
+ "SKY dataset. Atleast one column needs this dataset, or you "
+ "have asked to subtract the Sky from the values.\n\n"
+ "Please use the `--skyhdu' option to give a specific HDU "
+ "using its number (counting from zero) or name. If the "
+ "dataset is in another file, please use `--skyfile' to give "
+ "the filename", p->usedskyfile);
+
+ /* Read the sky image. */
+ p->sky=gal_fits_img_read_to_type(p->usedskyfile, p->skyhdu,
+ GAL_TYPE_FLOAT32, p->cp.minmapsize);
+
+ /* Check its size. */
+ ui_preparation_check_size_read_tiles(p, p->sky, p->usedskyfile,
+ p->skyhdu);
+
+ /* Subtract the Sky value. */
+ if(p->subtractsky && need_sky==0)
+ {
+ /* Subtract the Sky value. */
+ ui_subtract_sky(p->values, p->sky, &p->cp.tl);
+
+ /* If we don't need the Sky value, then free it here. */
+ if(need_sky==0)
+ {
+ gal_data_free(p->sky);
+ p->sky=NULL;
+ }
+ }
+ }
+ /* Read the Sky standard deviation image and check its size. */
+ if(need_std)
+ {
+ /* Make sure the HDU is also given. */
+ if(p->stdhdu==NULL)
+ error(EXIT_FAILURE, 0, "%s: no HDU/extension provided for the "
+ "SKY STANDARD DEVIATION dataset. Atleast one column needs "
+ "this dataset. Please use the `--stdhdu' option to give "
+ "a specific HDU using its number (counting from zero) or "
+ "name. If the dataset is in another file, please use "
+ "`--stdfile' to give the filename. If its actually the Sky's "
+ "variance dataset, run MakeCatalog with `--variance'",
+ p->usedstdfile);
+
+ /* Read the Sky standard deviation image into memory. */
+ p->std=gal_fits_img_read_to_type(p->usedstdfile, p->stdhdu,
+ GAL_TYPE_FLOAT32, p->cp.minmapsize);
+
+ /* Check its size. */
+ ui_preparation_check_size_read_tiles(p, p->std, p->usedstdfile,
+ p->stdhdu);
+ }
-/* To make the catalog processing more scalable (and later allow for
- over-lappping regions), we will define a tile for each object. */
-void
-ui_one_tile_per_object(struct mkcatalogparams *p)
-{
- size_t ndim=p->input->ndim;
- int32_t *l, *lf, *start;
- size_t i, d, *min, *max, width=2*ndim;
- size_t *minmax=gal_data_malloc_array(GAL_TYPE_SIZE_T,
- width*p->numobjects, __func__,
- "minmax");
- size_t *coord=gal_data_malloc_array(GAL_TYPE_SIZE_T, ndim, __func__,
- "coord");
+ /* Sanity checks on upper-limit measurements. */
+ if(p->upperlimit)
+ {
+ /* If an upperlimit check was requested, make sure the object number
+ is not larger than the maximum number of labels. */
+ if(p->checkupperlimit[0] != GAL_BLANK_INT32
+ && p->checkupperlimit[0] > p->numobjects)
+ error(EXIT_FAILURE, 0, "%d (object identifer for the "
+ "`--checkupperlimit' option) is larger than the number of "
+ "objects in the input labels (%zu)", p->checkupperlimit[0],
+ p->numobjects);
+
+ /* Read the mask file if it was given. */
+ if(p->upmaskfile)
+ {
+ /* Make sure the HDU for the mask image is given. */
+ if(p->upmaskhdu==NULL)
+ error(EXIT_FAILURE, 0, "%s: no HDU/extension provided, please "
+ "use the `--upmaskhdu' option to specify a specific HDU "
+ "using its number (counting from zero) or name",
+ p->upmaskfile);
+
+ /* Read the mask image. */
+ p->upmask = gal_fits_img_read(p->upmaskfile, p->upmaskhdu,
+ p->cp.minmapsize);
+
+ /* Check its size. */
+ if( gal_data_dsize_is_different(p->objects, p->upmask) )
+ error(EXIT_FAILURE, 0, "`%s' (hdu: %s) and `%s' (hdu: %s) have a"
+ "different dimension/size", p->upmaskfile, p->upmaskhdu,
+ p->objectsfile, p->cp.hdu);
+
+ /* If it isn't an integer type, report an error. */
+ if( p->upmask->type==GAL_TYPE_FLOAT32
+ || p->upmask->type==GAL_TYPE_FLOAT64 )
+ error(EXIT_FAILURE, 0, "%s (hdu: %s) has a %s numerical data "
+ "type. Only integer type inputs are acceptable as a mask."
+ "If the values are indeed integers, only placed in a "
+ "floating point container, you can use Gnuastro's "
+ "Arithmetic program to conver the numeric data type",
+ p->upmaskfile, p->upmaskhdu,
+ gal_type_name(p->upmask->type, 1));
+
+ /* Convert the mask to a uint8_t: with a 1 for all non-zero
+ pixels and 0 for zero pixels. */
+ zero=gal_data_alloc(NULL, GAL_TYPE_UINT8, 1, &one, NULL, 1, -1,
+ NULL, NULL, NULL);
+ p->upmask=gal_arithmetic(GAL_ARITHMETIC_OP_NE,
+ ( GAL_ARITHMETIC_INPLACE
+ | GAL_ARITHMETIC_FREE
+ | GAL_ARITHMETIC_NUMOK ),
+ p->upmask, zero);
+ }
+ }
+}
- /* Initialize the minimum and maximum position for each tile/object. So,
- we'll initialize the minimum coordinates to the maximum possible
- `size_t' value (in `GAL_BLANK_SIZE_T') and the maximums to zero. */
- for(i=0;i<p->numobjects;++i)
- for(d=0;d<ndim;++d)
- {
- minmax[ i * width + d ] = GAL_BLANK_SIZE_T; /* Minimum. */
- minmax[ i * width + ndim + d ] = 0; /* Maximum. */
- }
- /* Go over the objects label image and correct the minimum and maximum
- coordinates. */
- start=p->objects->array;
- lf=(l=p->objects->array)+p->objects->size;
- do
- if(*l>0)
- {
- /* Get the coordinates of this pixel. */
- gal_dimension_index_to_coord(l-start, ndim, p->objects->dsize, coord);
- /* Check to see this coordinate is the smallest/largest found so
- far for this label. Note that labels start from 1, while indexs
- here start from zero. */
- min = &minmax[ (*l-1) * width ];
- max = &minmax[ (*l-1) * width + ndim ];
- for(d=0;d<ndim;++d)
- {
- if( coord[d] < min[d] ) min[d] = coord[d];
- if( coord[d] > max[d] ) max[d] = coord[d];
- }
- }
- while(++l<lf);
- /* For a check.
- for(i=0;i<p->numobjects;++i)
- printf("%zu: (%zu, %zu) --> (%zu, %zu)\n", i+1, minmax[i*width],
- minmax[i*width+1], minmax[i*width+2], minmax[i*width+3]);
- */
+/* The necessary keywords from the objects or clumps image were read when
+ we were reading them. They were necessary during the
+ pre-processing. Here, we'll read the image from */
+static void
+ui_preparations_read_keywords(struct mkcatalogparams *p)
+{
+ float minstd;
+ gal_data_t *tmp;
+ gal_data_t *keys=NULL;
- /* Make the tiles. */
- p->tiles=gal_tile_series_from_minmax(p->input, minmax, p->numobjects);
+ if(p->std)
+ {
+ /* Read the keywords from the standard deviation image. */
+ keys=gal_data_array_calloc(2);
+ keys[0].next=&keys[1];
+ keys[0].name="MINSTD"; keys[1].name="MEDSTD";
+ keys[0].type=GAL_TYPE_FLOAT32; keys[1].type=GAL_TYPE_FLOAT32;
+ keys[0].array=&minstd; keys[1].array=&p->medstd;
+ gal_fits_key_read(p->usedstdfile, p->stdhdu, keys, 0, 0);
+
+ /* If the two keywords couldn't be read. We don't want to slow down
+ the user for the median (which needs sorting). So we'll just
+ calculate the minimum which is necessary for the `p->cpscorr'. */
+ if(keys[1].status) p->medstd=NAN;
+ if(keys[0].status)
+ {
+ /* Calculate the minimum STD. */
+ tmp=gal_statistics_minimum(p->std);
+ minstd=*((float *)(tmp->array));
+ gal_data_free(tmp);
- /* Clean up. */
- free(coord);
- free(minmax);
+ /* If the units are in variance, then take the square root. */
+ if(p->variance) minstd=sqrt(minstd);
+ }
+ p->cpscorr = minstd>1 ? 1.0f : minstd;
+
+ /* Clean up. */
+ keys[0].name=keys[1].name=NULL;
+ keys[0].array=keys[1].array=NULL;
+ gal_data_array_free(keys, 2, 1);
+ }
}
@@ -749,7 +1100,7 @@ ui_preparations_both_names(struct mkcatalogparams *p)
/* Note that suffix is not used in the text table outputs, so it
doesn't matter if the output table is not FITS. */
suffix="_catalog.fits";
- basename = p->inputname;
+ basename = p->objectsfile;
}
@@ -767,14 +1118,8 @@ ui_preparations_both_names(struct mkcatalogparams *p)
p->clumpsout=p->objectsout;
}
- /* Revert `keepinputdir' to what it was and free `p->cp.output', we will
- be using `p->objectsout' and `p->clumpsout' from now on. */
- if(p->cp.output)
- {
- p->cp.keepinputdir=keepinputdir;
- free(p->cp.output);
- p->cp.output=NULL;
- }
+ /* Revert `keepinputdir' to what it was. */
+ p->cp.keepinputdir=keepinputdir;
}
@@ -787,7 +1132,7 @@ ui_preparations_outnames(struct mkcatalogparams *p)
{
char *suffix;
- /* Set the output filename */
+ /* The process differs if an output filename has been given. */
if(p->cp.output)
{
/* If the output name is a FITS file, then
@@ -801,13 +1146,12 @@ ui_preparations_outnames(struct mkcatalogparams *p)
else
p->cp.tableformat=GAL_TABLE_FORMAT_TXT;
- /* If a clumps image has been read, then we have two outputs. */
+ /* If a clumps image is present, then we have two outputs. */
if(p->clumps) ui_preparations_both_names(p);
else
{
gal_checkset_writable_remove(p->cp.output, 0, p->cp.dontdelete);
- p->objectsout=p->cp.output;
- p->cp.output=NULL;
+ gal_checkset_allocate_copy(p->cp.output, &p->objectsout);
}
}
else
@@ -819,11 +1163,95 @@ ui_preparations_outnames(struct mkcatalogparams *p)
else
{
suffix = ( p->cp.tableformat==GAL_TABLE_FORMAT_TXT
- ? "._cat.txt" : "_cat.fits" );
- p->objectsout=gal_checkset_automatic_output(&p->cp, p->inputname,
+ ? "_cat.txt" : "_cat.fits" );
+ p->objectsout=gal_checkset_automatic_output(&p->cp, p->objectsfile,
suffix);
}
}
+
+ /* If an upperlimit check image is requsted, then set its filename. */
+ if(p->checkupperlimit)
+ {
+ suffix = ( p->cp.tableformat==GAL_TABLE_FORMAT_TXT
+ ? "_upcheck.txt" : "_upcheck.fits" );
+ p->upcheckout=gal_checkset_automatic_output(&p->cp,
+ ( p->cp.output
+ ? p->cp.output
+ : p->objectsfile),
+ suffix);
+ }
+
+ /* Just to avoid bugs (`p->cp.output' must no longer be used), we'll free
+ it and set it to NULL.*/
+ free(p->cp.output);
+ p->cp.output=NULL;
+}
+
+
+
+
+
+/* To make the catalog processing more scalable (and later allow for
+ over-lappping regions), we will define a tile for each object. */
+void
+ui_one_tile_per_object(struct mkcatalogparams *p)
+{
+ size_t ndim=p->objects->ndim;
+
+ int32_t *l, *lf, *start;
+ size_t i, d, *min, *max, width=2*ndim;
+ size_t *minmax=gal_data_malloc_array(GAL_TYPE_SIZE_T,
+ width*p->numobjects, __func__,
+ "minmax");
+ size_t *coord=gal_data_malloc_array(GAL_TYPE_SIZE_T, ndim, __func__,
+ "coord");
+
+
+ /* Initialize the minimum and maximum position for each tile/object. So,
+ we'll initialize the minimum coordinates to the maximum possible
+ `size_t' value (in `GAL_BLANK_SIZE_T') and the maximums to zero. */
+ for(i=0;i<p->numobjects;++i)
+ for(d=0;d<ndim;++d)
+ {
+ minmax[ i * width + d ] = GAL_BLANK_SIZE_T; /* Minimum. */
+ minmax[ i * width + ndim + d ] = 0; /* Maximum. */
+ }
+
+ /* Go over the objects label image and correct the minimum and maximum
+ coordinates. */
+ start=p->objects->array;
+ lf=(l=p->objects->array)+p->objects->size;
+ do
+ if(*l>0)
+ {
+ /* Get the coordinates of this pixel. */
+ gal_dimension_index_to_coord(l-start, ndim, p->objects->dsize, coord);
+
+ /* Check to see this coordinate is the smallest/largest found so
+ far for this label. Note that labels start from 1, while indexs
+ here start from zero. */
+ min = &minmax[ (*l-1) * width ];
+ max = &minmax[ (*l-1) * width + ndim ];
+ for(d=0;d<ndim;++d)
+ {
+ if( coord[d] < min[d] ) min[d] = coord[d];
+ if( coord[d] > max[d] ) max[d] = coord[d];
+ }
+ }
+ while(++l<lf);
+
+ /* For a check.
+ for(i=0;i<p->numobjects;++i)
+ printf("%zu: (%zu, %zu) --> (%zu, %zu)\n", i+1, minmax[i*width],
+ minmax[i*width+1], minmax[i*width+2], minmax[i*width+3]);
+ */
+
+ /* Make the tiles. */
+ p->tiles=gal_tile_series_from_minmax(p->objects, minmax, p->numobjects);
+
+ /* Clean up. */
+ free(coord);
+ free(minmax);
}
@@ -841,9 +1269,9 @@ ui_preparations_upperlimit(struct mkcatalogparams *p)
if(p->uprange)
{
for(i=0;p->uprange[i]!=-1;++i) ++c;
- if(c!=p->input->ndim)
+ if(c!=p->objects->ndim)
error(EXIT_FAILURE, 0, "%zu values given to `--uprange', but input "
- "has %zu dimensions", c, p->input->ndim);
+ "has %zu dimensions", c, p->objects->ndim);
}
/* Check the number of random samples. */
@@ -896,28 +1324,28 @@ ui_preparations(struct mkcatalogparams *p)
error(EXIT_FAILURE, 0, "no columns requested, please run again with "
"`--help' for the full list of columns you can ask for");
+ /* Set the actual filenames to use. */
+ ui_set_filenames(p);
+
+ /* Read the main input (the objects image). */
+ ui_read_labels(p);
+
+ /* Prepare the output columns. */
+ columns_define_alloc(p);
/* Read the inputs. */
ui_preparations_read_inputs(p);
-
/* Read the helper keywords from the inputs and if they aren't present
then calculate the necessary parameters. */
ui_preparations_read_keywords(p);
-
/* Set the output filename(s). */
ui_preparations_outnames(p);
-
- /* Allocate the output columns to fill up with the program. */
- columns_define_alloc(p);
-
-
/* Make the tiles that cover each object. */
ui_one_tile_per_object(p);
-
/* Allocate the reference random number generator and seed values. It
will be cloned once for every thread. If the user hasn't called
`envseed', then we want it to be different for every run, so we need
@@ -954,7 +1382,6 @@ void
ui_read_check_inputs_setup(int argc, char *argv[], struct mkcatalogparams *p)
{
struct gal_options_common_params *cp=&p->cp;
- char *skyfile, *stdfile, *clumpsfile, *objectsfile;
/* Include the parameters necessary for argp from this program (`args.h')
@@ -983,9 +1410,9 @@ ui_read_check_inputs_setup(int argc, char *argv[], struct
mkcatalogparams *p)
/* Read the options into the program's structure, and check them and
- their relations prior to printing.
+ their relations prior to printing. */
ui_read_check_only_options(p);
- */
+
/* Print the option values if asked. Note that this needs to be done
after the option checks so un-sane values are not printed in the
@@ -1006,22 +1433,21 @@ ui_read_check_inputs_setup(int argc, char *argv[],
struct mkcatalogparams *p)
/* Inform the user. */
if(!p->cp.quiet)
{
- /* Set the names for easy reading. */
- skyfile = p->skyfile ? p->skyfile : p->inputname;
- stdfile = p->stdfile ? p->stdfile : p->inputname;
- clumpsfile = p->clumpsfile ? p->clumpsfile : p->inputname;
- objectsfile = p->objectsfile ? p->objectsfile : p->inputname;
-
/* Write the information. */
printf(PROGRAM_NAME" started on %s", ctime(&p->rawtime));
printf(" - Using %zu CPU thread%s\n", p->cp.numthreads,
p->cp.numthreads==1 ? "." : "s.");
- printf(" - Input: %s (hdu: %s)\n", p->inputname, p->cp.hdu);
- printf(" - Objects: %s (hdu: %s)\n", objectsfile, p->objectshdu);
+ printf(" - Objects: %s (hdu: %s)\n", p->objectsfile, p->cp.hdu);
if(p->clumps)
- printf(" - Clumps: %s (hdu: %s)\n", clumpsfile, p->clumpshdu);
- printf(" - Sky: %s (hdu: %s)\n", skyfile, p->skyhdu);
- printf(" - Sky STD: %s (hdu: %s)\n", stdfile, p->stdhdu);
+ printf(" - Clumps: %s (hdu: %s)\n", p->usedclumpsfile,
+ p->clumpshdu);
+ if(p->values)
+ printf(" - Values: %s (hdu: %s)\n", p->usedvaluesfile,
+ p->valueshdu);
+ if(p->subtractsky || p->sky)
+ printf(" - Sky: %s (hdu: %s)\n", p->usedskyfile, p->skyhdu);
+ if(p->std)
+ printf(" - Sky STD: %s (hdu: %s)\n", p->usedstdfile, p->stdhdu);
if(p->upmaskfile)
printf(" - Upper limit magnitude mask: %s (hdu: %s)\n",
p->upmaskfile, p->cp.hdu);
@@ -1071,7 +1497,7 @@ ui_free_report(struct mkcatalogparams *p, struct timeval
*t1)
/* Free the types of the WCS coordinates (for catalog meta-data). */
if(p->ctype)
{
- for(d=0;d<p->input->ndim;++d)
+ for(d=0;d<p->objects->ndim;++d)
free(p->ctype[d]);
free(p->ctype);
}
@@ -1093,17 +1519,38 @@ ui_free_report(struct mkcatalogparams *p, struct
timeval *t1)
free(p->skyfile);
free(p->stdfile);
free(p->clumpshdu);
- free(p->objectshdu);
+ free(p->valueshdu);
free(p->clumpsfile);
- free(p->objectsfile);
+ free(p->valuesfile);
gal_data_free(p->sky);
gal_data_free(p->std);
- gal_data_free(p->input);
+ gal_data_free(p->values);
gal_data_free(p->upmask);
gal_data_free(p->clumps);
gal_data_free(p->objects);
+ if(p->upcheckout) free(p->upcheckout);
gal_data_array_free(p->tiles, p->numobjects, 0);
+ /* If the Sky or its STD image were given in tiles, then we defined a
+ tile structure to deal with them. The initialization of the tile
+ structure is checked with its `ndim' element. */
+ if(p->cp.tl.ndim) gal_tile_full_free_contents(&p->cp.tl);
+
+ /* If an upper limit range warning is necessary, print it here. */
+ if(p->uprangewarning)
+ fprintf(stderr, "\nMore on the WARNING-UPPERLIMIT(s) above: "
+ "In order to obtain a good/robust random distribution (and "
+ "thus a reliable upper-limit measurement), it is necessary "
+ "to have a sufficienty wide enough range (in each dimension). "
+ "As mentioned in the warning(s) above, the available "
+ "range for random sampling of some of the labels in this "
+ "input is less than double their length. If the input is taken "
+ "from a larger dataset, this issue can be solved by using a "
+ "larger part of it. You can also run MakeCatalog with "
+ "`--checkupperlimit' to see the distribution for a special "
+ "object or clump as a table and personally inspect its "
+ "reliability. \n\n");
+
/* Print the final message. */
if(!p->cp.quiet)
gal_timing_report(t1, PROGRAM_NAME" finished in: ", 0);
diff --git a/bin/mkcatalog/ui.h b/bin/mkcatalog/ui.h
index 77fca48..75b1309 100644
--- a/bin/mkcatalog/ui.h
+++ b/bin/mkcatalog/ui.h
@@ -47,18 +47,17 @@ enum program_args_groups
/* Available letters for short options:
- f g k l u v w x y z
- H J L W X Y
+ f g k w x y z
+ E H J L O R W X Y
*/
enum option_keys_enum
{
/* With short-option version. */
- UI_KEY_OBJECTSFILE = 'O', /* General settings. */
- UI_KEY_CLUMPSFILE = 'C',
+ UI_KEY_CLUMPSCAT = 'C', /* General settings. */
+ UI_KEY_VALUESFILE = 'v',
+ UI_KEY_CLUMPSFILE = 'l',
UI_KEY_SKYFILE = 's',
UI_KEY_STDFILE = 't',
- UI_KEY_SKYSUBTRACTED = 'E',
- UI_KEY_THRESHOLD = 'R',
UI_KEY_ENVSEED = 'e',
UI_KEY_IDS = 'i', /* Catalog columns. */
@@ -81,12 +80,14 @@ enum option_keys_enum
/* Only with long version (start with a value 1000, the rest will be set
automatically). */
- UI_KEY_OBJECTSHDU = 1000, /* General settings. */
+ UI_KEY_VALUESHDU = 1000, /* General settings. */
UI_KEY_CLUMPSHDU,
UI_KEY_SKYHDU,
UI_KEY_STDHDU,
+ UI_KEY_WITHCLUMPS,
UI_KEY_ZEROPOINT,
UI_KEY_VARIANCE,
+ UI_KEY_SUBTRACTSKY,
UI_KEY_SFMAGNSIGMA,
UI_KEY_SFMAGAREA,
UI_KEY_UPMASKFILE,
@@ -95,11 +96,13 @@ enum option_keys_enum
UI_KEY_UPRANGE,
UI_KEY_UPSIGMACLIP,
UI_KEY_UPNSIGMA,
+ UI_KEY_CHECKUPPERLIMIT,
UI_KEY_OBJID, /* Catalog columns. */
UI_KEY_IDINHOSTOBJ,
UI_KEY_CLUMPSAREA,
UI_KEY_WEIGHTAREA,
+ UI_KEY_GEOAREA,
UI_KEY_GEOX,
UI_KEY_GEOY,
UI_KEY_CLUMPSX,
diff --git a/bin/mkcatalog/upperlimit.c b/bin/mkcatalog/upperlimit.c
index 11790d3..0a30f60 100644
--- a/bin/mkcatalog/upperlimit.c
+++ b/bin/mkcatalog/upperlimit.c
@@ -27,6 +27,7 @@ along with Gnuastro. If not, see
<http://www.gnu.org/licenses/>.
#include <error.h>
#include <float.h>
#include <stdlib.h>
+#include <inttypes.h>
#include <gnuastro/tile.h>
#include <gnuastro/threads.h>
@@ -46,14 +47,13 @@ along with Gnuastro. If not, see
<http://www.gnu.org/licenses/>.
static gal_data_t *
upperlimit_make_clump_tiles(struct mkcatalog_passparams *pp)
{
- gal_data_t *input=pp->p->input;
- size_t ndim=input->ndim, *tsize=pp->tile->dsize;
+ gal_data_t *objects=pp->p->objects;
+ size_t ndim=objects->ndim, *tsize=pp->tile->dsize;
- int32_t *O, *C;
gal_data_t *tiles=NULL;
- float *I, *II, *start=input->array;
size_t increment=0, num_increment=1;
size_t i, d, *min, *max, width=2*ndim;
+ int32_t *O, *OO, *C, *start=objects->array;
size_t *coord=gal_data_malloc_array(GAL_TYPE_SIZE_T, ndim, __func__,
"coord");
size_t *minmax=gal_data_malloc_array(GAL_TYPE_SIZE_T,
@@ -75,19 +75,17 @@ upperlimit_make_clump_tiles(struct mkcatalog_passparams *pp)
while( pp->start_end_inc[0] + increment <= pp->start_end_inc[1] )
{
/* Set the pointers for this tile. */
- I = pp->st_i + increment;
- O = pp->st_o + increment;
- C = pp->st_c + increment;
+ C = pp->st_c + increment;
+ OO = ( O = pp->st_o + increment ) + tsize[ndim-1];
/* Go over the contiguous region. */
- II = I + tsize[ndim-1];
do
{
/* Only consider clumps. */
if( *O==pp->object && *C>0 )
{
/* Get the coordinates of this pixel. */
- gal_dimension_index_to_coord(I-start, ndim, input->dsize,
+ gal_dimension_index_to_coord(O-start, ndim, objects->dsize,
coord);
/* Check to see if this coordinate is the smallest/largest
@@ -103,12 +101,12 @@ upperlimit_make_clump_tiles(struct mkcatalog_passparams
*pp)
}
/* Increment the other pointers. */
- ++O; ++C;
+ ++C;
}
- while(++I<II);
+ while(++O<OO);
/* Increment to the next contiguous region. */
- increment += ( gal_tile_block_increment(input, tsize, num_increment++,
+ increment += ( gal_tile_block_increment(objects, tsize, num_increment++,
NULL) );
}
@@ -119,7 +117,7 @@ upperlimit_make_clump_tiles(struct mkcatalog_passparams *pp)
*/
/* Make the tiles. */
- tiles=gal_tile_series_from_minmax(input, minmax, pp->clumpsinobj);
+ tiles=gal_tile_series_from_minmax(objects, minmax, pp->clumpsinobj);
/* Cleanup and return. */
free(coord);
@@ -149,19 +147,21 @@ upperlimit_make_clump_tiles(struct mkcatalog_passparams
*pp)
/*********************************************************************/
/******************* For one tile ********************/
/*********************************************************************/
+/* Set the minimum and maximum possible range to place the FIRST pixel of
+ the object/clump tile over the dataset. */
static void
upperlimit_random_range(struct mkcatalog_passparams *pp, gal_data_t *tile,
size_t *min, size_t *max, int32_t clumplab)
{
struct mkcatalogparams *p=pp->p;
size_t d, tstart, minext, maxext, coord[]={0,0};
- size_t ndim=p->input->ndim, *dsize=p->input->dsize;
+ size_t ndim=p->objects->ndim, *dsize=p->objects->dsize;
/* Set the minimum and maximum acceptable value for the range. */
if(p->uprange)
{
tstart=gal_data_ptr_dist(tile->block->array, tile->array,
- p->input->type);
+ p->objects->type);
gal_dimension_index_to_coord(tstart, ndim, dsize, coord);
}
@@ -211,25 +211,27 @@ upperlimit_random_range(struct mkcatalog_passparams *pp,
gal_data_t *tile,
}
else
{
+ /* We are positioning the FIRST pixel of the tile, not the
+ center. So, the minimum possible value is zero, and in order
+ to not push out of the image, the maximum is the
+ `tile->dsize[d]' away from the edge. */
min[d]=0;
max[d]=dsize[d]-tile->dsize[d]-1;
}
- /* A small sanity check. */
+ /* A small warning to the user if the range isn't large enough. */
if( max[d]-min[d] < 2*tile->dsize[d] )
{
+ p->uprangewarning=1;
if(clumplab)
- fprintf(stderr, "WARNING: object %d clump %d: range of random "
- "positions (%zu) along dimension %zu for upper-limit "
- "calculations is smaller than double of its size (%zu) "
- "in this dimension.\n\n", pp->object, clumplab,
- max[d]-min[d], ndim-d, 2*tile->dsize[d]);
+ fprintf(stderr, "WARNING-UPPERLIMIT: object %d clump %d, "
+ "dimension %zu: range (%zu) < 2*size (%zu).\n",
+ pp->object, clumplab, ndim-d, max[d]-min[d],
+ 2*tile->dsize[d]);
else
- fprintf(stderr, "WARNING: object %d: range of random "
- "positions (%zu) along dimension %zu for upper-limit "
- "calculations is smaller than double of its size (%zu) "
- "in this dimension.\n\n", pp->object, max[d]-min[d],
- ndim-d, 2*tile->dsize[d]);
+ fprintf(stderr, "WARNING-UPPERLIMIT: object %d, dimension %zu: "
+ "range (%zu) < 2*size (%zu).\n", pp->object, ndim-d,
+ max[d]-min[d], 2*tile->dsize[d]);
}
}
}
@@ -250,7 +252,7 @@ upperlimit_random_position(struct mkcatalog_passparams *pp,
gal_data_t *tile,
maximum of the particular generator. It may happen that the labeled
region extends the full range of a dimension. In that case, the only
possible starting point would be 0. */
- if( (int)(p->input->dsize[dim]) - (int)(tile->dsize[dim]) > 0 )
+ if( (int)(p->objects->dsize[dim]) - (int)(tile->dsize[dim]) > 0 )
{
r=gsl_rng_get(pp->rng); /* For easy reading. */
return lrint( (float)(min[dim])
@@ -265,6 +267,172 @@ upperlimit_random_position(struct mkcatalog_passparams
*pp, gal_data_t *tile,
+/* It is necessary to write the upperlimit parameters into the output
+ tables. The same set of information will thus be necessary both in the
+ upperlimit check table and also the final output. This function will do
+ the job in both cases.
+
+ Note that in the check output, the sigma-clipping information is not
+ used/necessary, so to avoid confusion, we won't write it.
+*/
+void
+upperlimit_write_comments(struct mkcatalogparams *p,
+ gal_list_str_t **comments, int withsigclip)
+{
+ char *str;
+
+ if(p->cp.tableformat==GAL_TABLE_FORMAT_TXT)
+ {
+ if(asprintf(&str, "--------- Upper-limit measurement ---------")<0)
+ error(EXIT_FAILURE, 0, "%s: asprintf allocation", __func__);
+ gal_list_str_add(comments, str, 0);
+ }
+
+ if( asprintf(&str, "Number of usable random samples: %zu", p->upnum)<0 )
+ error(EXIT_FAILURE, 0, "%s: asprintf allocation", __func__);
+ gal_list_str_add(comments, str, 0);
+
+ if(p->uprange)
+ {
+ if( asprintf(&str, "Range of random samples about target: %zu, %zu",
+ p->uprange[1], p->uprange[0])<0 )
+ error(EXIT_FAILURE, 0, "%s: asprintf allocation", __func__);
+ gal_list_str_add(comments, str, 0);
+ }
+
+ if( asprintf(&str, "Random number generator name: %s", p->rngname)<0 )
+ error(EXIT_FAILURE, 0, "%s: asprintf allocation", __func__);
+ gal_list_str_add(comments, str, 0);
+
+ if( asprintf(&str, "Random number generator seed: %"PRIu64, p->seed)<0 )
+ error(EXIT_FAILURE, 0, "%s: asprintf allocation", __func__);
+ gal_list_str_add(comments, str, 0);
+
+ if(withsigclip)
+ {
+ if( asprintf(&str, "Multiple of STD used for sigma-clipping: %.3f",
+ p->upsigmaclip[0])<0 )
+ error(EXIT_FAILURE, 0, "%s: asprintf allocation", __func__);
+ gal_list_str_add(comments, str, 0);
+
+ if(p->upsigmaclip[1]>=1.0f)
+ {
+ if( asprintf(&str, "Number of clips for sigma-clipping: %.0f",
+ p->upsigmaclip[1])<0 )
+ error(EXIT_FAILURE, 0, "%s: asprintf allocation", __func__);
+ }
+ else
+ {
+ if( asprintf(&str, "Tolerance level to sigma-clipping: %.3f",
+ p->upsigmaclip[1])<0 )
+ error(EXIT_FAILURE, 0, "%s: asprintf allocation", __func__);
+ }
+ gal_list_str_add(comments, str, 0);
+
+ if( p->oiflag[ OCOL_UPPERLIMIT_B ] )
+ {
+ if( asprintf(&str, "Multiple of sigma-clipped STD for upper-limit: "
+ "%.3f", p->upnsigma)<0 )
+ error(EXIT_FAILURE, 0, "%s: asprintf allocation", __func__);
+ gal_list_str_add(comments, str, 0);
+ }
+ }
+}
+
+
+
+
+
+/* Write the values into a table for the user */
+static void
+upperlimit_write_check(struct mkcatalogparams *p, gal_list_sizet_t *check_x,
+ gal_list_sizet_t *check_y, gal_list_f32_t *check_s)
+{
+ float *sarr;
+ gal_data_t *x, *y, *s;
+ char *tmp=NULL, *tmp2=NULL;
+ gal_list_str_t *comments=NULL;
+ size_t *xarr, *yarr, tnum, num;
+
+
+ /* Convert the lists to an array. */
+ xarr=gal_list_sizet_to_array(check_x, 1, &num);
+ yarr=gal_list_sizet_to_array(check_y, 1, &tnum);
+ if(tnum!=num)
+ error(EXIT_FAILURE, 0, "%s: a bug! Please contact us at %s to fix the "
+ "problem. For some reason the size of the input lists don't "
+ "match (%zu, %zu)", __func__, PACKAGE_BUGREPORT, tnum, num);
+ sarr=gal_list_f32_to_array(check_s, 1, &tnum);
+ if(tnum!=num)
+ error(EXIT_FAILURE, 0, "%s: a bug! Please contact us at %s to fix the "
+ "problem. For some reason the size of the input lists don't "
+ "match (%zu, %zu)", __func__, PACKAGE_BUGREPORT, tnum, num);
+
+
+ /* Put the arrays into a data container. */
+ x=gal_data_alloc(xarr, GAL_TYPE_SIZE_T, 1, &num, NULL, 0, p->cp.minmapsize,
+ "RANDOM_X", "pixel",
+ "X-axis position of random footprint's first pixel.");
+ y=gal_data_alloc(yarr, GAL_TYPE_SIZE_T, 1, &num, NULL, 0, p->cp.minmapsize,
+ "RANDOM_Y", "pixel",
+ "Y-axis position of random footprint's first pixel.");
+ s=gal_data_alloc(sarr, GAL_TYPE_FLOAT32, 1, &num, NULL, 0, p->cp.minmapsize,
+ "RANDOM_SUM",
+ p->values->unit ? p->values->unit : "input-units",
+ "Sum of pixel values over random footprint.");
+
+
+ /* Convert the unsigned 64-bit values to 32-bit because the FITS table
+ format doesn't recognize 64-bit integers.*/
+ x=gal_data_copy_to_new_type_free(x, GAL_TYPE_UINT32);
+ y=gal_data_copy_to_new_type_free(y, GAL_TYPE_UINT32);
+
+
+ /* Write exactly what object/clump this table is for. */
+ if( p->checkupperlimit[1]!=GAL_BLANK_INT32 )
+ if( asprintf(&tmp2, ", Clump %d", p->checkupperlimit[1]) <0 )
+ error(EXIT_FAILURE, 0, "%s: asprintf allocation", __func__);
+ if( asprintf(&tmp, "Upperlimit distribution for Object %d%s",
+ p->checkupperlimit[0],
+ ( p->checkupperlimit[1]==GAL_BLANK_INT32
+ ? "" : tmp2) ) <0 )
+ error(EXIT_FAILURE, 0, "%s: asprintf allocation", __func__);
+ gal_list_str_add(&comments, tmp, 0);
+ if(tmp2) {free(tmp2); tmp2=NULL;}
+
+
+ /* Write the basic info, and conclude the comments. */
+ mkcatalog_write_inputs_in_comments(p, &comments, 0, 0);
+ upperlimit_write_comments(p, &comments, 0);
+ if(p->cp.tableformat==GAL_TABLE_FORMAT_TXT)
+ {
+ if( asprintf(&tmp, "--------- Table columns ---------")<0 )
+ error(EXIT_FAILURE, 0, "%s: asprintf allocation", __func__);
+ gal_list_str_add(&comments, tmp, 0);
+ }
+
+
+ /* Define a list from the containers and write them into a table. */
+ x->next=y;
+ y->next=s;
+ gal_list_str_reverse(&comments);
+ gal_table_write(x, comments, p->cp.tableformat, p->upcheckout,
+ "UPPERLIMIT_CHECK");
+
+ /* Inform the user. */
+ if(!p->cp.quiet)
+ printf(" - Upperlimit check table: %s\n", p->upcheckout);
+
+ /* Clean up. */
+ gal_data_free(x);
+ gal_data_free(y);
+ gal_data_free(s);
+}
+
+
+
+
+
/* Given the distribution of values, do the upper-limit calculations. */
static void
upperlimit_measure(struct mkcatalog_passparams *pp, int32_t clumplab,
@@ -355,8 +523,8 @@ upperlimit_measure(struct mkcatalog_passparams *pp, int32_t
clumplab,
}
else
{
- o[ clumplab ? CCOL_UPPERLIMIT_S : OCOL_UPPERLIMIT_S ] = NAN;
o[ clumplab ? CCOL_UPPERLIMIT_B : OCOL_UPPERLIMIT_B ] = NAN;
+ o[ clumplab ? CCOL_UPPERLIMIT_S : OCOL_UPPERLIMIT_S ] = NAN;
o[ clumplab ? CCOL_UPPERLIMIT_Q : OCOL_UPPERLIMIT_Q ] = NAN;
}
}
@@ -370,21 +538,37 @@ upperlimit_one_tile(struct mkcatalog_passparams *pp,
gal_data_t *tile,
unsigned long seed, int32_t clumplab)
{
struct mkcatalogparams *p=pp->p;
- size_t ndim=p->input->ndim, *dsize=p->input->dsize;
+ size_t ndim=p->objects->ndim, *dsize=p->objects->dsize;
double sum;
void *tarray;
- int continueparse;
uint8_t *M=NULL, *st_m=NULL;
- float *uparr=pp->up_vals->array;
- float *I, *II, *SK, *st_i, *st_sky;
+ int continueparse, writecheck=0;
+ struct gal_list_f32_t *check_s=NULL;
+ float *V, *st_v, *uparr=pp->up_vals->array;
size_t d, tcounter=0, counter=0, se_inc[2];
size_t min[2], max[2], increment, num_increment;
- int32_t *O, *oO, *st_o, *st_oo, *st_oc, *oC=NULL;
+ struct gal_list_sizet_t *check_x=NULL, *check_y=NULL;
+ int32_t *O, *OO, *oO, *st_o, *st_oo, *st_oc, *oC=NULL;
size_t maxcount = p->upnum * MKCATALOG_UPPERLIMIT_STOP_MULTIP;
size_t *rcoord=gal_data_malloc_array(GAL_TYPE_SIZE_T, ndim, __func__,
"rcoord");
+ /* See if a check table must be created for this distribution. */
+ if( p->checkupperlimit[0]==pp->object )
+ {
+ /* We are on a clump */
+ if( clumplab )
+ {
+ if( p->checkupperlimit[1]==clumplab )
+ writecheck=1;
+ }
+ else
+ if( p->checkupperlimit[1]==GAL_BLANK_INT32 )
+ writecheck=1;
+ }
+
+
/* Initializations. */
tarray=tile->array;
gsl_rng_set(pp->rng, seed);
@@ -410,9 +594,9 @@ upperlimit_one_tile(struct mkcatalog_passparams *pp,
gal_data_t *tile,
rcoord[d] = upperlimit_random_position(pp, tile, d, min, max);
/* Set the tile's new starting pointer. */
- tile->array = gal_data_ptr_increment(p->input->array,
+ tile->array = gal_data_ptr_increment(p->objects->array,
gal_dimension_coord_to_index(ndim, dsize, rcoord),
- p->input->type);
+ p->objects->type);
/* Starting and ending coordinates for this random position, note
that in `pp' we have the starting and ending coordinates of the
@@ -423,65 +607,83 @@ upperlimit_one_tile(struct mkcatalog_passparams *pp,
gal_data_t *tile,
sum = 0.0f;
/* Starting pointers for the random tile. */
- st_i = gal_tile_start_end_ind_inclusive(tile, p->input, se_inc);
+ st_v = gal_tile_start_end_ind_inclusive(tile, p->values, se_inc);
st_o = (int32_t *)(p->objects->array) + se_inc[0];
- st_sky = (float *)(p->sky->array) + se_inc[0];
if(p->upmask) st_m = (uint8_t *)(p->upmask->array) + se_inc[0];
/* Parse over this object/clump. */
while( se_inc[0] + increment <= se_inc[1] )
{
/* Set the pointers. */
- I = st_i + increment; /* Random tile. */
- SK = st_sky + increment; /* Random tile. */
- O = st_o + increment; /* Random tile. */
- if(st_m) M = st_m + increment; /* Random tile. */
- oO = st_oo + increment; /* Original tile. */
- if(clumplab) oC = st_oc + increment; /* Original tile. */
+ V = st_v + increment; /* Random tile. */
+ O = st_o + increment; /* Random tile. */
+ if(st_m) M = st_m + increment; /* Random tile. */
+ oO = st_oo + increment; /* Original tile. */
+ if(clumplab) oC = st_oc + increment; /* Original tile. */
/* Parse over this contiguous region, similar to the first and
second pass functions. */
- II = I + tile->dsize[ndim-1];
+ OO = O + tile->dsize[ndim-1];
do
{
/* Only use pixels over this object/clump. */
- if( *oO==pp->object
- && ( oC==NULL || clumplab==0 || *oC==clumplab ) )
+ if( *oO==pp->object && ( oC==NULL || *oC==clumplab ) )
{
- if( *O || (M && *M) || ( p->hasblank && isnan(*I) ) )
+ /* If this pixel is a non-zero object code, or is masked,
+ or has a blank value, then stop parsing. */
+ if( *O || (M && *M) || ( p->hasblank && isnan(*V) ) )
continueparse=0;
else
- sum += *I-*SK;
+ sum += *V;
}
/* Increment the other pointers. */
- ++SK; ++O; ++oO; if(oC) ++oC;
+ ++V;
+ ++oO;
+ if(M) ++M;
+ if(oC) ++oC;
}
- while(continueparse && ++I<II);
+ while(continueparse && ++O<OO);
/* Increment to the next contiguous region of this tile. */
if(continueparse)
- increment += ( gal_tile_block_increment(p->input, dsize,
+ increment += ( gal_tile_block_increment(p->objects, dsize,
num_increment++, NULL) );
else break;
}
- /* Further processing is only necessary if this random tile
- actually covered the sky region. */
+ /* Further processing is only necessary if this random tile was fully
+ parsed. */
if(continueparse) uparr[ counter++ ] = sum;
+ /* If a check is necessary, write in the values. */
+ if(writecheck)
+ {
+ gal_list_sizet_add(&check_x, rcoord[1]+1);
+ gal_list_sizet_add(&check_y, rcoord[0]+1);
+ gal_list_f32_add(&check_s, continueparse ? sum : NAN);
+ }
+
+
/* Increment the total-counter. */
++tcounter;
}
+ /* If a check is necessary, then write the values. */
+ if(writecheck)
+ upperlimit_write_check(p, check_x, check_y, check_s);
+
/* Do the measurement on the random distribution. */
upperlimit_measure(pp, clumplab, counter==p->upnum);
/* Reset the tile's array pointer, clean up and return. */
tile->array=tarray;
free(rcoord);
+ gal_list_f32_free(check_s);
+ gal_list_sizet_free(check_x);
+ gal_list_sizet_free(check_y);
}
@@ -522,6 +724,17 @@ upperlimit_calculate(struct mkcatalog_passparams *pp)
within this object. */
if(p->clumps && pp->clumpsinobj)
{
+ /* If an upper-limit check image is requested, then make sure that
+ the clump label is not more than the number of clumps in this
+ object. */
+ if( p->checkupperlimit[0] == pp->object
+ && p->checkupperlimit[1] != GAL_BLANK_INT32
+ && p->checkupperlimit[1] > pp->clumpsinobj )
+ error(EXIT_FAILURE, 0, "object %d has %zu clumps, but an upperlimit "
+ "check table (using the `--checkupperlimit' option) has been "
+ "requested for clump %d", pp->object, pp->clumpsinobj,
+ p->checkupperlimit[1]);
+
/* Make tiles covering the clumps. */
clumptiles=upperlimit_make_clump_tiles(pp);
diff --git a/bin/mkcatalog/upperlimit.h b/bin/mkcatalog/upperlimit.h
index 143252f..001ed83 100644
--- a/bin/mkcatalog/upperlimit.h
+++ b/bin/mkcatalog/upperlimit.h
@@ -24,6 +24,10 @@ along with Gnuastro. If not, see
<http://www.gnu.org/licenses/>.
#define UPPERLIMIT_H
void
+upperlimit_write_comments(struct mkcatalogparams *p,
+ gal_list_str_t **comments, int withsigclip);
+
+void
upperlimit_calculate(struct mkcatalog_passparams *pp);
#endif
diff --git a/bin/noisechisel/Makefile.am b/bin/noisechisel/Makefile.am
index 3c0b937..4bf46c3 100644
--- a/bin/noisechisel/Makefile.am
+++ b/bin/noisechisel/Makefile.am
@@ -30,11 +30,11 @@ bin_PROGRAMS = astnoisechisel
astnoisechisel_LDADD = -lgnuastro
-astnoisechisel_SOURCES = main.c ui.c clumps.c detection.c noisechisel.c \
- sky.c segmentation.c threshold.c
+astnoisechisel_SOURCES = main.c ui.c detection.c noisechisel.c sky.c \
+ threshold.c
-EXTRA_DIST = main.h authors-cite.h args.h ui.h clumps.h detection.h \
- noisechisel.h segmentation.h sky.h threshold.h
+EXTRA_DIST = main.h authors-cite.h args.h ui.h detection.h noisechisel.h \
+ sky.h threshold.h
diff --git a/bin/noisechisel/args.h b/bin/noisechisel/args.h
index 10a72a7..0f3f033 100644
--- a/bin/noisechisel/args.h
+++ b/bin/noisechisel/args.h
@@ -1,5 +1,5 @@
/*********************************************************************
-NoiseChisel - Detect and segment signal in a noisy dataset.
+NoiseChisel - Detect signal in a noisy dataset.
NoiseChisel is part of GNU Astronomy Utilities (Gnuastro) package.
Original author:
@@ -73,13 +73,13 @@ struct argp_option program_options[] =
GAL_OPTIONS_NOT_SET
},
{
- "convolvedhdu",
- UI_KEY_CONVOLVEDHDU,
+ "chdu",
+ UI_KEY_CHDU,
"STR",
0,
"HDU/extension of convolved image in file.",
GAL_OPTIONS_GROUP_INPUT,
- &p->convolvedhdu,
+ &p->chdu,
GAL_TYPE_STRING,
GAL_OPTIONS_RANGE_ANY,
GAL_OPTIONS_NOT_MANDATORY,
@@ -99,57 +99,20 @@ struct argp_option program_options[] =
GAL_OPTIONS_NOT_SET
},
{
- "wkhdu",
- UI_KEY_WKHDU,
+ "whdu",
+ UI_KEY_WHDU,
"STR",
0,
"HDU containing wide kernel image.",
GAL_OPTIONS_GROUP_INPUT,
- &p->wkhdu,
+ &p->whdu,
GAL_TYPE_STRING,
GAL_OPTIONS_RANGE_ANY,
GAL_OPTIONS_NOT_MANDATORY,
GAL_OPTIONS_NOT_SET
},
- {
- "skysubtracted",
- UI_KEY_SKYSUBTRACTED,
- 0,
- 0,
- "Input is Sky subtracted (for error estimation).",
- GAL_OPTIONS_GROUP_INPUT,
- &p->skysubtracted,
- GAL_OPTIONS_NO_ARG_TYPE,
- GAL_OPTIONS_RANGE_0_OR_1,
- GAL_OPTIONS_NOT_MANDATORY,
- GAL_OPTIONS_NOT_SET
- },
- {
- "minskyfrac",
- UI_KEY_MINSKYFRAC,
- "FLT",
- 0,
- "Min. fraction of undetected area in tile.",
- GAL_OPTIONS_GROUP_INPUT,
- &p->minskyfrac,
- GAL_TYPE_FLOAT32,
- GAL_OPTIONS_RANGE_GE_0_LE_1,
- GAL_OPTIONS_MANDATORY,
- GAL_OPTIONS_NOT_SET
- },
- {
- "minnumfalse",
- UI_KEY_MINNUMFALSE,
- "INT",
- 0,
- "Minimum number for S/N estimation.",
- GAL_OPTIONS_GROUP_INPUT,
- &p->minnumfalse,
- GAL_TYPE_SIZE_T,
- GAL_OPTIONS_RANGE_GT_0,
- GAL_OPTIONS_MANDATORY,
- GAL_OPTIONS_NOT_SET
- },
+
+
/* Tessellation. */
@@ -174,13 +137,13 @@ struct argp_option program_options[] =
/* Output options. */
{
- "onlydetection",
- UI_KEY_ONLYDETECTION,
+ "rawoutput",
+ UI_KEY_RAWOUTPUT,
0,
0,
- "Stop at the end of detection.",
+ "Output only detection labels & 1-elem/tile grid.",
GAL_OPTIONS_GROUP_OUTPUT,
- &p->onlydetection,
+ &p->rawoutput,
GAL_OPTIONS_NO_ARG_TYPE,
GAL_OPTIONS_RANGE_0_OR_1,
GAL_OPTIONS_NOT_MANDATORY,
@@ -190,7 +153,6 @@ struct argp_option program_options[] =
-
/* Detection. */
{
0, 0, 0, 0,
@@ -355,6 +317,19 @@ struct argp_option program_options[] =
gal_options_read_sigma_clip
},
{
+ "minskyfrac",
+ UI_KEY_MINSKYFRAC,
+ "FLT",
+ 0,
+ "Min. fraction of undetected area in tile.",
+ UI_GROUP_DETECTION,
+ &p->minskyfrac,
+ GAL_TYPE_FLOAT32,
+ GAL_OPTIONS_RANGE_GE_0_LE_1,
+ GAL_OPTIONS_MANDATORY,
+ GAL_OPTIONS_NOT_SET
+ },
+ {
"checkdetsky",
UI_KEY_CHECKDETSKY,
0,
@@ -381,39 +356,52 @@ struct argp_option program_options[] =
GAL_OPTIONS_NOT_SET
},
{
- "detsnminarea",
- UI_KEY_DETSNMINAREA,
+ "snminarea",
+ UI_KEY_SNMINAREA,
"INT",
0,
"Min. pseudo-detection area for S/N dist.",
UI_GROUP_DETECTION,
- &p->detsnminarea,
+ &p->snminarea,
GAL_TYPE_SIZE_T,
GAL_OPTIONS_RANGE_ANY,
GAL_OPTIONS_MANDATORY,
GAL_OPTIONS_NOT_SET
},
{
- "checkdetsn",
- UI_KEY_CHECKDETSN,
+ "checksn",
+ UI_KEY_CHECKSN,
0,
0,
"Save pseudo-detection S/N values to a file.",
UI_GROUP_DETECTION,
- &p->checkdetsn,
+ &p->checksn,
GAL_OPTIONS_NO_ARG_TYPE,
GAL_OPTIONS_RANGE_0_OR_1,
GAL_OPTIONS_NOT_MANDATORY,
GAL_OPTIONS_NOT_SET
},
{
- "detquant",
- UI_KEY_DETQUANT,
+ "minnumfalse",
+ UI_KEY_MINNUMFALSE,
+ "INT",
+ 0,
+ "Minimum number for S/N estimation.",
+ UI_GROUP_DETECTION,
+ &p->minnumfalse,
+ GAL_TYPE_SIZE_T,
+ GAL_OPTIONS_RANGE_GT_0,
+ GAL_OPTIONS_MANDATORY,
+ GAL_OPTIONS_NOT_SET
+ },
+ {
+ "snquant",
+ UI_KEY_SNQUANT,
"FLT",
0,
"Quantile in pseudo-det. to define true.",
UI_GROUP_DETECTION,
- &p->detquant,
+ &p->snquant,
GAL_TYPE_FLOAT32,
GAL_OPTIONS_RANGE_GT_0_LT_1,
GAL_OPTIONS_MANDATORY,
@@ -489,134 +477,6 @@ struct argp_option program_options[] =
- /* Segmentation */
- {
- 0, 0, 0, 0,
- "Segmentation:",
- UI_GROUP_SEGMENTATION
- },
- {
- "segsnminarea",
- UI_KEY_SEGSNMINAREA,
- "INT",
- 0,
- "Minimum area of clumps for S/N estimation.",
- UI_GROUP_SEGMENTATION,
- &p->segsnminarea,
- GAL_TYPE_SIZE_T,
- GAL_OPTIONS_RANGE_GT_0,
- GAL_OPTIONS_MANDATORY,
- GAL_OPTIONS_NOT_SET
- },
- {
- "checkclumpsn",
- UI_KEY_CHECKCLUMPSN,
- 0,
- 0,
- "Save Sky clump S/N values into a file.",
- UI_GROUP_SEGMENTATION,
- &p->checkclumpsn,
- GAL_OPTIONS_NO_ARG_TYPE,
- GAL_OPTIONS_RANGE_0_OR_1,
- GAL_OPTIONS_NOT_MANDATORY,
- GAL_OPTIONS_NOT_SET
- },
- {
- "segquant",
- UI_KEY_SEGQUANT,
- "FLT",
- 0,
- "S/N Quantile of true sky clumps.",
- UI_GROUP_SEGMENTATION,
- &p->segquant,
- GAL_TYPE_FLOAT32,
- GAL_OPTIONS_RANGE_GT_0,
- GAL_OPTIONS_MANDATORY,
- GAL_OPTIONS_NOT_SET
- },
- {
- "keepmaxnearriver",
- UI_KEY_KEEPMAXNEARRIVER,
- 0,
- 0,
- "Keep clumps with peak touching a river.",
- UI_GROUP_SEGMENTATION,
- &p->keepmaxnearriver,
- GAL_OPTIONS_NO_ARG_TYPE,
- GAL_OPTIONS_RANGE_0_OR_1,
- GAL_OPTIONS_NOT_MANDATORY,
- GAL_OPTIONS_NOT_SET
- },
- {
- "gthresh",
- UI_KEY_GTHRESH,
- "FLT",
- 0,
- "Multiple of STD to stop growing clumps.",
- UI_GROUP_SEGMENTATION,
- &p->gthresh,
- GAL_TYPE_FLOAT32,
- GAL_OPTIONS_RANGE_ANY,
- GAL_OPTIONS_MANDATORY,
- GAL_OPTIONS_NOT_SET
- },
- {
- "minriverlength",
- UI_KEY_MINRIVERLENGTH,
- "INT",
- 0,
- "Minimum len of useful grown clump rivers.",
- UI_GROUP_SEGMENTATION,
- &p->minriverlength,
- GAL_TYPE_SIZE_T,
- GAL_OPTIONS_RANGE_ANY,
- GAL_OPTIONS_MANDATORY,
- GAL_OPTIONS_NOT_SET
- },
- {
- "objbordersn",
- UI_KEY_OBJBORDERSN,
- "FLT",
- 0,
- "Min. S/N for grown clumps as one object.",
- UI_GROUP_SEGMENTATION,
- &p->objbordersn,
- GAL_TYPE_FLOAT32,
- GAL_OPTIONS_RANGE_GE_0,
- GAL_OPTIONS_MANDATORY,
- GAL_OPTIONS_NOT_SET
- },
- {
- "grownclumps",
- UI_KEY_GROWNCLUMPS,
- 0,
- 0,
- "Save grown clumps instead of original.",
- UI_GROUP_SEGMENTATION,
- &p->grownclumps,
- GAL_OPTIONS_NO_ARG_TYPE,
- GAL_OPTIONS_RANGE_0_OR_1,
- GAL_OPTIONS_NOT_MANDATORY,
- GAL_OPTIONS_NOT_SET
- },
- {
- "checksegmentation",
- UI_KEY_CHECKSEGMENTATION,
- 0,
- 0,
- "Store segmentation steps in a file.",
- UI_GROUP_SEGMENTATION,
- &p->checksegmentation,
- GAL_OPTIONS_NO_ARG_TYPE,
- GAL_OPTIONS_RANGE_0_OR_1,
- GAL_OPTIONS_NOT_MANDATORY,
- GAL_OPTIONS_NOT_SET
- },
-
-
-
-
-
/* Operating mode options. */
{
"continueaftercheck",
diff --git a/bin/noisechisel/astnoisechisel.conf
b/bin/noisechisel/astnoisechisel.conf
index 71bdd47..320e74f 100644
--- a/bin/noisechisel/astnoisechisel.conf
+++ b/bin/noisechisel/astnoisechisel.conf
@@ -19,8 +19,8 @@
# Input:
khdu 1
- wkhdu 1
- convolvedhdu 1
+ whdu 1
+ chdu 1
minskyfrac 0.7
minnumfalse 100
@@ -40,18 +40,10 @@
openingngb 8
sigmaclip 3,0.2
dthresh 0.0
- detsnminarea 10
- detquant 0.95
+ snminarea 10
+ snquant 0.95
detgrowquant 0.70
detgrowmaxholesize 100
-# Segmentation
- segsnminarea 15
- keepmaxnearriver 0
- segquant 0.95
- gthresh 0.5
- minriverlength 15
- objbordersn 1
-
# Operating mode
continueaftercheck 0
diff --git a/bin/noisechisel/authors-cite.h b/bin/noisechisel/authors-cite.h
index 782d42d..81fe4b2 100644
--- a/bin/noisechisel/authors-cite.h
+++ b/bin/noisechisel/authors-cite.h
@@ -1,5 +1,5 @@
/*********************************************************************
-NoiseChisel - Detect and segment signal in a noisy dataset.
+NoiseChisel - Detect signal in a noisy dataset.
NoiseChisel is part of GNU Astronomy Utilities (Gnuastro) package.
Original author:
diff --git a/bin/noisechisel/detection.c b/bin/noisechisel/detection.c
index c7e08a7..7203038 100644
--- a/bin/noisechisel/detection.c
+++ b/bin/noisechisel/detection.c
@@ -1,9 +1,9 @@
/*********************************************************************
-NoiseChisel - Detect and segment signal in a noisy dataset.
+NoiseChisel - Detect signal in a noisy dataset.
NoiseChisel is part of GNU Astronomy Utilities (Gnuastro) package.
Original author:
- Mohammad Akhlaghi <address@hidden>
+ Mohammad Akhlaghi <address@hidden>
Contributing author(s):
Copyright (C) 2015-2018, Free Software Foundation, Inc.
@@ -29,6 +29,7 @@ along with Gnuastro. If not, see
<http://www.gnu.org/licenses/>.
#include <string.h>
#include <gnuastro/fits.h>
+#include <gnuastro/label.h>
#include <gnuastro/binary.h>
#include <gnuastro/threads.h>
#include <gnuastro/dimension.h>
@@ -40,7 +41,6 @@ along with Gnuastro. If not, see
<http://www.gnu.org/licenses/>.
#include "ui.h"
#include "sky.h"
-#include "clumps.h"
#include "threshold.h"
@@ -495,7 +495,7 @@ detection_sn(struct noisechiselparams *p, gal_data_t
*worklab, size_t num,
sn = gal_data_alloc(NULL, GAL_TYPE_FLOAT32, 1, &tablen, NULL, 1,
p->cp.minmapsize, "SIGNAL-TO-NOISE", "ratio",
NULL);
- snind = ( p->checkdetsn==0 ? NULL
+ snind = ( p->checksn==0 ? NULL
: gal_data_alloc(NULL, GAL_TYPE_INT32, 1, &tablen, NULL, 1,
p->cp.minmapsize, "LABEL", "counter",
NULL) );
@@ -560,7 +560,7 @@ detection_sn(struct noisechiselparams *p, gal_data_t
*worklab, size_t num,
plabend = (plab=worklab->array) + worklab->size;
do
if( *plab!=GAL_BLANK_INT32
- && ( area[*plab]<p->detsnminarea || brightness[*plab]<0) )
+ && ( area[*plab]<p->snminarea || brightness[*plab]<0) )
*plab=0;
while(++plab<plabend);
}
@@ -577,7 +577,7 @@ detection_sn(struct noisechiselparams *p, gal_data_t
*worklab, size_t num,
for(i=1;i<tablen;++i)
{
ave=brightness[i]/area[i];
- if( area[i]>p->detsnminarea && ave>0.0f && xy[i*xyncols]>0.0f )
+ if( area[i]>p->snminarea && ave>0.0f && xy[i*xyncols]>0.0f )
{
/* Get the flux weighted center coordinates. */
coord[0]=GAL_DIMENSION_FLT_TO_INT( xy[i*xyncols+1]/xy[i*xyncols] );
@@ -589,10 +589,6 @@ detection_sn(struct noisechiselparams *p, gal_data_t
*worklab, size_t num,
err = ((float *)(p->std->array))[
gal_tile_full_id_from_coord(&p->cp.tl, coord) ];
- /* If the image was already sky subtracted, the second power of
- the error needs to be doubled. */
- err *= p->skysubtracted ? err : 2.0f*err;
-
/* Correct the index in the sn to store the Signal to noise
ratio. When we are dealing with the noise, we only want the
non-zero signal to noise values, so we will just use a
@@ -602,7 +598,7 @@ detection_sn(struct noisechiselparams *p, gal_data_t
*worklab, size_t num,
ind = s0d1D2 ? i : counter++;
if(snind) indarr[ind]=i;
snarr[ind] = ( sqrt( (float)(area[i])/p->cpscorr )
- * ave / sqrt(ave+err) );
+ * ave / sqrt( ave + err*err ) );
}
else
/* In detection pseudo-detections, order matters, so we will set
@@ -720,19 +716,19 @@ detection_pseudo_real(struct noisechiselparams *p)
error(EXIT_FAILURE, 0, "only %zu pseudo-detections could be found over "
"the sky region to estimate an S/N. This is less than %zu (value "
"to `--minnumfalse' option). Please adjust parameters like "
- "`--dthresh', `--detsnminarea', or make sure that there actually "
+ "`--dthresh', `--snminarea', or make sure that there actually "
"is sufficient sky area after initial detection. You can use "
"`--checkdetection' to see every step until this point", sn->size,
p->minnumfalse);
/* Get the S/N quantile and report it if we are in non-quiet mode. */
- quant=gal_statistics_quantile(sn, p->detquant, 1);
+ quant=gal_statistics_quantile(sn, p->snquant, 1);
p->detsnthresh = *((float *)(quant->array));
if(!p->cp.quiet)
{
- if( asprintf(&msg, "Pseudo-det S/N: %.2f (%.2f quant of %zu).",
- p->detsnthresh, p->detquant, sn->size)<0 )
+ if( asprintf(&msg, "Pseudo-det S/N: %.3f (%g quant of %zu).",
+ p->detsnthresh, p->snquant, sn->size)<0 )
error(EXIT_FAILURE, 0, "%s: asprintf allocation", __func__);
gal_timing_report(&t1, msg, 2);
free(msg);
@@ -993,7 +989,7 @@ detection_quantile_expand(struct noisechiselparams *p,
gal_data_t *workbin)
/* If the binary value is 1, then we want an initial label of 1
(the object is already detected). If it isn't, then we only
want it if it is above the threshold. */
- *o = *b==1 ? 1 : ( *arr>*e_th ? CLUMPS_INIT : 0);
+ *o = *b==1 ? 1 : ( *arr>*e_th ? GAL_LABEL_INIT : 0);
if(*b==0 && *arr>*e_th)
*d++ = o - (int32_t *)(p->olabel->array);
@@ -1005,7 +1001,7 @@ detection_quantile_expand(struct noisechiselparams *p,
gal_data_t *workbin)
while(++o<of);
/* Expand the detections. */
- clumps_grow(p->olabel, diffuseindexs, 0, p->olabel->ndim);
+ gal_label_grow_indexs(p->olabel, diffuseindexs, 0, p->olabel->ndim);
/* Only keep the 1 valued pixels in the binary array and fill its
holes. */
diff --git a/bin/noisechisel/detection.h b/bin/noisechisel/detection.h
index df679b1..fedfcf6 100644
--- a/bin/noisechisel/detection.h
+++ b/bin/noisechisel/detection.h
@@ -1,5 +1,5 @@
/*********************************************************************
-NoiseChisel - Detect and segment signal in a noisy dataset.
+NoiseChisel - Detect signal in a noisy dataset.
NoiseChisel is part of GNU Astronomy Utilities (Gnuastro) package.
Original author:
diff --git a/bin/noisechisel/main.c b/bin/noisechisel/main.c
index c49c91b..00479e9 100644
--- a/bin/noisechisel/main.c
+++ b/bin/noisechisel/main.c
@@ -1,5 +1,5 @@
/*********************************************************************
-NoiseChisel - Detect and segment signal in a noisy dataset.
+NoiseChisel - Detect signal in a noisy dataset.
NoiseChisel is part of GNU Astronomy Utilities (Gnuastro) package.
Original author:
diff --git a/bin/noisechisel/main.h b/bin/noisechisel/main.h
index ed78ef1..0cf1711 100644
--- a/bin/noisechisel/main.h
+++ b/bin/noisechisel/main.h
@@ -1,5 +1,5 @@
/*********************************************************************
-NoiseChisel - Detect and segment signal in a noisy dataset.
+NoiseChisel - Detect signal in a noisy dataset.
NoiseChisel is part of GNU Astronomy Utilities (Gnuastro) package.
Original author:
@@ -45,24 +45,21 @@ struct noisechiselparams
struct gal_tile_two_layer_params ltl;/* Large tessellation. */
char *inputname; /* Input filename. */
char *kernelname; /* Input kernel filename. */
- char *widekernelname; /* Name of wider kernel to be used. */
- char *convolvedname; /* Convolved image (to avoid convolution).*/
- char *convolvedhdu; /* HDU of convolved image. */
char *khdu; /* Kernel HDU. */
- char *wkhdu; /* Wide kernel HDU. */
- uint8_t skysubtracted; /* Input has been Sky subtracted before. */
- float minskyfrac; /* Undetected area min. frac. in tile. */
- size_t minnumfalse; /* Min No. of det/seg for true quantile. */
+ char *convolvedname; /* Convolved image (to avoid convolution).*/
+ char *chdu; /* HDU of convolved image. */
+ char *widekernelname; /* Name of wider kernel to be used. */
+ char *whdu; /* Wide kernel HDU. */
- uint8_t onlydetection; /* Do not do any segmentation. */
- uint8_t grownclumps; /* Save grown clumps instead of original. */
uint8_t continueaftercheck; /* Don't abort after the check steps. */
+ uint8_t rawoutput; /* Only detection & 1 elem/tile output. */
float mirrordist; /* Maximum distance to check mode sym. */
float modmedqdiff; /* Difference between mode and median. */
+ float minskyfrac; /* Undetected area min. frac. in tile. */
float qthresh; /* Quantile threshold on convolved image. */
float qthreshtilequant; /* Remove tiles with lower quantile. */
- size_t smoothwidth; /* Width of flat kernel to smooth. */
+ size_t smoothwidth; /* Interpolation: flat kernel to smooth. */
uint8_t checkqthresh; /* Save the quantile threhsold steps. */
size_t erode; /* Number of erosions after thresholding. */
size_t erodengb; /* Connectivity for erosion. */
@@ -72,24 +69,16 @@ struct noisechiselparams
double sigmaclip[2]; /* Sigma-clipping parameters. */
uint8_t checkdetsky; /* Check pseudo-detection sky value. */
float dthresh; /* Sigma threshold for Pseudo-detections. */
- size_t detsnminarea; /* Minimum pseudo-detection area for S/N. */
- uint8_t checkdetsn; /* Save pseudo-detection S/N values. */
- float detquant; /* True detection quantile. */
+ size_t snminarea; /* Minimum pseudo-detection area for S/N. */
+ uint8_t checksn; /* Save pseudo-detection S/N values. */
+ size_t minnumfalse; /* Min No. of det/seg for true quantile. */
+ float snquant; /* True detection quantile. */
float detgrowquant; /* Quantile to grow true detections. */
size_t detgrowmaxholesize; /* Max. size of holes to fill in growth. */
uint8_t cleangrowndet; /* Remove grown objects with small S/N. */
uint8_t checkdetection; /* Save all detection steps to a file. */
uint8_t checksky; /* Check the Sky value estimation. */
- size_t segsnminarea; /* Minimum area for segmentation. */
- uint8_t checkclumpsn; /* Save the clump S/N values to a file. */
- float segquant; /* Quantile of clumps in sky for true S/N.*/
- uint8_t keepmaxnearriver; /* Keep clumps with a peak near a river. */
- float gthresh; /* Multiple of STD to stop growing clumps.*/
- size_t minriverlength; /* Min, len of good grown clump rivers. */
- float objbordersn; /* Minimum S/N for grown clumps to be one.*/
- uint8_t checksegmentation; /* Save the segmentation steps in file. */
-
/* Internal. */
char *qthreshname; /* Name of Quantile threshold check image.*/
char *detskyname; /* Name of Initial det sky check image. */
@@ -98,9 +87,6 @@ struct noisechiselparams
char *detsn_D_name; /* Final detection S/N name. */
char *detectionname; /* Name of detection steps file. */
char *skyname; /* Name of Sky estimation steps file. */
- char *clumpsn_s_name; /* Sky clump S/N name. */
- char *clumpsn_d_name; /* Detection clumps S/N name. */
- char *segmentationname; /* Name of segmentation steps file. */
gal_data_t *input; /* Input image. */
gal_data_t *kernel; /* Sharper kernel. */
@@ -109,7 +95,6 @@ struct noisechiselparams
gal_data_t *wconv; /* Convolved with wider kernel. */
gal_data_t *binary; /* For binary operations. */
gal_data_t *olabel; /* Labels of objects in the detection. */
- gal_data_t *clabel; /* Labels of clumps in the detection. */
gal_data_t *expand_thresh; /* Quantile threshold to expand per tile. */
gal_data_t *exp_thresh_full; /* Full array containing growth thresh. */
gal_data_t *sky; /* Mean of undetected pixels, per tile. */
@@ -128,10 +113,7 @@ struct noisechiselparams
size_t numinitialdets; /* Number of initial detections. */
size_t numdetections; /* Number of final detections. */
- size_t numclumps; /* Number of true clumps. */
- size_t numobjects; /* Number of objects. */
float detsnthresh; /* Pseudo-detection S/N threshold. */
- float clumpsnthresh; /* Clump S/N threshold. */
};
#endif
diff --git a/bin/noisechisel/noisechisel.c b/bin/noisechisel/noisechisel.c
index 4e5656f..8f27ce9 100644
--- a/bin/noisechisel/noisechisel.c
+++ b/bin/noisechisel/noisechisel.c
@@ -1,5 +1,5 @@
/*********************************************************************
-NoiseChisel - Detect and segment signal in a noisy dataset.
+NoiseChisel - Detect signal in a noisy dataset.
NoiseChisel is part of GNU Astronomy Utilities (Gnuastro) package.
Original author:
@@ -40,7 +40,6 @@ along with Gnuastro. If not, see
<http://www.gnu.org/licenses/>.
#include "sky.h"
#include "detection.h"
#include "threshold.h"
-#include "segmentation.h"
@@ -119,52 +118,6 @@ noisechisel_convolve(struct noisechiselparams *p)
-static void
-noisechisel_find_sky_subtract(struct noisechiselparams *p)
-{
- /* Find the final Sky value. */
- sky_and_std(p, p->skyname);
-
- /* Abort if the user only wanted to see until this point.*/
- if(p->skyname && !p->continueaftercheck)
- ui_abort_after_check(p, p->skyname, NULL,
- "derivation of final Sky (and its STD) value");
-
- /* Subtract the Sky from the Input and Convolved (necessary for
- segmentation) images. */
- sky_subtract(p);
-}
-
-
-
-
-
-/* If convolution was not done while respecting channel edges (when there
- is more than one channel, pixels outside the edge weren't used in the
- convolution), then correct it. */
-static void
-noisechisel_convolve_correct_ch_edges(struct noisechiselparams *p)
-{
- struct gal_tile_two_layer_params *tl=&p->cp.tl;
-
- /* Correct the convolved image if necessary. */
- if( tl->totchannels>1 && tl->workoverch==0 )
- {
- /* Do the correction. */
- gal_convolve_spatial_correct_ch_edge(tl->tiles, p->kernel,
- p->cp.numthreads, 1, p->conv);
-
- /* Inform the user. */
- if(!p->cp.quiet)
- gal_timing_report(NULL, "Corrected convolution of touching channel "
- "edges", 1);
- }
-}
-
-
-
-
-
@@ -183,94 +136,40 @@ noisechisel_convolve_correct_ch_edges(struct
noisechiselparams *p)
/***********************************************************************/
/************* Output ***************/
/***********************************************************************/
-/* The input image has been sky subtracted for further processing. So we'll
- need to copy the input image directly into the output. */
-static void
-noisechisel_output_copy_input(struct noisechiselparams *p)
-{
- int status=0;
- fitsfile *in, *out;
- char card[FLEN_CARD];
-
-
- /* Create/open the output file. */
- out=gal_fits_open_to_write(p->cp.output);
-
-
- /* Open the input FITS file in the proper extension. */
- in=gal_fits_hdu_open(p->inputname, p->cp.hdu, READWRITE);
-
-
- /* Copy the input HDU into the output. */
- if( fits_copy_hdu(in, out, 0, &status) )
- gal_fits_io_error(status, "copying input hdu into first output hdu");
-
-
- /* If an extension name exists in the input HDU, then don't touch it. If
- the input doesn't have any, then make an EXTNAME keyword for it. Note
- that `fits_read_card' will return a non-zero if it doesn't find the
- keyword. */
- if( fits_read_card(out, "EXTNAME", card, &status) )
- {
- status=0;
- fits_write_key(out, TSTRING, "EXTNAME", "INPUT", "", &status);
- }
-
-
- /* Close the two files. */
- fits_close_file(in, &status);
- fits_close_file(out, &status);
-}
-
-
-
-
-
/* Write the output file. */
static void
noisechisel_output(struct noisechiselparams *p)
{
gal_fits_list_key_t *keys=NULL;
- /* Copy the input image into the first extension. */
- noisechisel_output_copy_input(p);
+
+ /* Put a copy of the input into the output (when necessary). */
+ if(p->rawoutput==0)
+ {
+ /* Subtract the Sky value. */
+ sky_subtract(p);
+
+ /* Correct the name of the input and write it out. */
+ if(p->input->name) free(p->input->name);
+ p->input->name="INPUT-NO-SKY";
+ gal_fits_img_write(p->input, p->cp.output, NULL, PROGRAM_NAME);
+ p->input->name=NULL;
+ }
/* Write the object labels and useful information into it's header. */
- if(p->onlydetection==0)
- gal_fits_key_list_add(&keys, GAL_TYPE_STRING, "WCLUMPS", 0, "yes", 0,
- "Generate catalog with clumps?", 0, "bool");
gal_fits_key_list_add(&keys, GAL_TYPE_SIZE_T, "NUMLABS", 0,
- &p->numobjects, 0, "Total number of labels "
+ &p->numdetections, 0, "Total number of labels "
"(inclusive)", 0, "counter");
gal_fits_key_list_add(&keys, GAL_TYPE_FLOAT32, "DETSN", 0, &p->detsnthresh,
0, "Minimum S/N of true pseudo-detections", 0,
"ratio");
- p->olabel->name = p->onlydetection ? "DETECTIONS" : "OBJECTS";
+ p->olabel->name = "DETECTIONS";
gal_fits_img_write(p->olabel, p->cp.output, keys, PROGRAM_NAME);
p->olabel->name=NULL;
keys=NULL;
- /* Write the clumps labels and useful information into it's header. Note
- that to make the clumps image more easily viewable, we will set all
- sky pixels to blank. Only clump pixels that have an overlapping object
- pixel will be use anyway, so the sky pixels are irrelevant. */
- if(p->onlydetection==0)
- {
- p->clabel->name="CLUMPS";
- gal_fits_key_list_add(&keys, GAL_TYPE_SIZE_T, "NUMLABS", 0,
- &p->numclumps, 0, "Total number of clumps", 0,
- "counter");
- gal_fits_key_list_add(&keys, GAL_TYPE_FLOAT32, "CLUMPSN", 0,
- &p->clumpsnthresh, 0, "Minimum S/N of true clumps",
- 0, "ratio");
- gal_fits_img_write(p->clabel, p->cp.output, keys, PROGRAM_NAME);
- p->clabel->name=NULL;
- keys=NULL;
- }
-
-
/* Write the Sky image into the output */
if(p->sky->name) free(p->sky->name);
p->sky->name="SKY";
@@ -331,19 +230,13 @@ noisechisel(struct noisechiselparams *p)
input and convolved images. */
if(p->numdetections)
{
- noisechisel_find_sky_subtract(p);
-
- /* If the user only wanted detection, ignore the segmentation steps. */
- if(p->onlydetection==0)
- {
- /* Correct the convolved image channel edges if necessary. */
- noisechisel_convolve_correct_ch_edges(p);
+ /* Find the final Sky and Sky STD values. */
+ sky_and_std(p, p->skyname);
- /* Do the segmentation. */
- segmentation(p);
- }
- else
- p->numobjects=p->numdetections;
+ /* Abort if the user only wanted to see until this point.*/
+ if(p->skyname && !p->continueaftercheck)
+ ui_abort_after_check(p, p->skyname, NULL,
+ "derivation of final Sky (and its STD) value");
/* Write the output. */
noisechisel_output(p);
diff --git a/bin/noisechisel/noisechisel.h b/bin/noisechisel/noisechisel.h
index 6fb6459..72376e5 100644
--- a/bin/noisechisel/noisechisel.h
+++ b/bin/noisechisel/noisechisel.h
@@ -1,5 +1,5 @@
/*********************************************************************
-NoiseChisel - Detect and segment signal in a noisy dataset.
+NoiseChisel - Detect signal in a noisy dataset.
NoiseChisel is part of GNU Astronomy Utilities (Gnuastro) package.
Original author:
diff --git a/bin/noisechisel/sky.c b/bin/noisechisel/sky.c
index b6e3ab5..ca0f35b 100644
--- a/bin/noisechisel/sky.c
+++ b/bin/noisechisel/sky.c
@@ -1,5 +1,5 @@
/*********************************************************************
-NoiseChisel - Detect and segment signal in a noisy dataset.
+NoiseChisel - Detect signal in a noisy dataset.
NoiseChisel is part of GNU Astronomy Utilities (Gnuastro) package.
Original author:
@@ -239,9 +239,8 @@ void
sky_subtract(struct noisechiselparams *p)
{
size_t tid;
- void *tarray=NULL;
+ gal_data_t *tile;
float *sky=p->sky->array;
- gal_data_t *tile, *tblock=NULL;
/* A small sanity check. */
if(p->sky->type!=GAL_TYPE_FLOAT32)
@@ -255,31 +254,7 @@ sky_subtract(struct noisechiselparams *p)
/* For easy reading. */
tile=&p->cp.tl.tiles[tid];
- /* First subtract the Sky value from the input image. */
+ /* Subtract the Sky value from the input image. */
GAL_TILE_PARSE_OPERATE(tile, NULL, 0, 0, {*i-=sky[tid];});
-
- /* Change to the convolved image (if there is any). */
- if(p->conv!=p->input)
- {
- tarray=tile->array;
- tblock=tile->block;
- tile->array=gal_tile_block_relative_to_other(tile, p->conv);
- tile->block=p->conv;
-
- /* The threshold is always low. So for the majority of non-NaN
- pixels in the image, the condition above will be true. If we
- come over a NaN pixel, then by definition of NaN, all
- conditionals will fail.
-
- If an image doesn't have any NaN pixels, only the pixels below
- the threshold have to be checked for a NaN which are by
- definition a very small fraction of the total pixels. And if
- there are NaN pixels in the image. */
- GAL_TILE_PARSE_OPERATE(tile, NULL, 0, 0, {*i-=sky[tid];});
-
- /* Revert back to the original block. */
- tile->array=tarray;
- tile->block=tblock;
- }
}
}
diff --git a/bin/noisechisel/sky.h b/bin/noisechisel/sky.h
index faf6502..eaf0b73 100644
--- a/bin/noisechisel/sky.h
+++ b/bin/noisechisel/sky.h
@@ -1,5 +1,5 @@
/*********************************************************************
-NoiseChisel - Detect and segment signal in a noisy dataset.
+NoiseChisel - Detect signal in a noisy dataset.
NoiseChisel is part of GNU Astronomy Utilities (Gnuastro) package.
Original author:
diff --git a/bin/noisechisel/threshold.c b/bin/noisechisel/threshold.c
index 09f66a9..1168efd 100644
--- a/bin/noisechisel/threshold.c
+++ b/bin/noisechisel/threshold.c
@@ -1,5 +1,5 @@
/*********************************************************************
-NoiseChisel - Detect and segment signal in a noisy dataset.
+NoiseChisel - Detect signal in a noisy dataset.
NoiseChisel is part of GNU Astronomy Utilities (Gnuastro) package.
Original author:
diff --git a/bin/noisechisel/threshold.h b/bin/noisechisel/threshold.h
index 66b18c4..daa6dfc 100644
--- a/bin/noisechisel/threshold.h
+++ b/bin/noisechisel/threshold.h
@@ -1,5 +1,5 @@
/*********************************************************************
-NoiseChisel - Detect and segment signal in a noisy dataset.
+NoiseChisel - Detect signal in a noisy dataset.
NoiseChisel is part of GNU Astronomy Utilities (Gnuastro) package.
Original author:
diff --git a/bin/noisechisel/ui.c b/bin/noisechisel/ui.c
index 87cd718..6e60cca 100644
--- a/bin/noisechisel/ui.c
+++ b/bin/noisechisel/ui.c
@@ -1,5 +1,5 @@
/*********************************************************************
-NoiseChisel - Detect and segment signal in a noisy dataset.
+NoiseChisel - Detect signal in a noisy dataset.
NoiseChisel is part of GNU Astronomy Utilities (Gnuastro) package.
Original author:
@@ -217,8 +217,8 @@ ui_read_check_only_options(struct noisechiselparams *p)
{
/* If the convolved option is given, then the convolved HDU is also
mandatory. */
- if(p->convolvedname && p->convolvedhdu==NULL)
- error(EXIT_FAILURE, 0, "no value given to `--convolvedhdu'. When the "
+ if(p->convolvedname && p->chdu==NULL)
+ error(EXIT_FAILURE, 0, "no value given to `--chdu'. When the "
"`--convolved' option is called (to specify a convolved image "
"and avoid convolution) it is mandatory to also specify a HDU "
"for it");
@@ -241,9 +241,9 @@ ui_read_check_only_options(struct noisechiselparams *p)
/* For the options that make tables, the table formation option is
mandatory. */
- if( (p->checkdetsn || p->checkclumpsn) && p->cp.tableformat==0 )
+ if( p->checksn && p->cp.tableformat==0 )
error(EXIT_FAILURE, 0, "`--tableformat' is necessary with the "
- "`--checkdetsn' and `--checkclumpsn' options.\n"
+ "`--checksn' option.\n"
"Please see description for `--tableformat' after running the "
"following command for more information (use `SPACE' to go down "
"the page and `q' to return to the command-line):\n\n"
@@ -271,12 +271,12 @@ ui_read_check_only_options(struct noisechiselparams *p)
gal_checkset_check_file(p->widekernelname);
/* If its FITS, see if a HDU has been provided. */
- if( gal_fits_name_is_fits(p->widekernelname) && p->wkhdu==NULL )
- error(EXIT_FAILURE, 0, "no HDU specified for wide kernel. When the "
- "wide kernel is a FITS file, a HDU must also be specified. You "
- "can use the `--khdu' option and give it the HDU number "
- "(starting from zero), extension name, or anything "
- "acceptable by CFITSIO");
+ if( gal_fits_name_is_fits(p->widekernelname) && p->whdu==NULL )
+ error(EXIT_FAILURE, 0, "no HDU specified for the given wide kernel "
+ "(`%s'). When the wide kernel is a FITS file, a HDU must also "
+ "be specified. You can use the `--whdu' option and give it the "
+ "HDU number (starting from zero), extension name, or any "
+ "HDU identifier acceptable by CFITSIO", p->widekernelname);
}
}
@@ -364,7 +364,7 @@ ui_set_output_names(struct noisechiselparams *p)
"_detsky.fits");
/* Pseudo-detection S/N values. */
- if(p->checkdetsn)
+ if(p->checksn)
{
p->detsn_s_name=gal_checkset_automatic_output(&p->cp, basename,
( p->cp.tableformat==GAL_TABLE_FORMAT_TXT
@@ -380,27 +380,11 @@ ui_set_output_names(struct noisechiselparams *p)
/* Detection steps. */
if(p->checkdetection)
p->detectionname=gal_checkset_automatic_output(&p->cp, basename,
- "_det.fits");
+ "_detcheck.fits");
- /* Detection steps. */
+ /* Sky checks. */
if(p->checksky)
p->skyname=gal_checkset_automatic_output(&p->cp, basename, "_sky.fits");
-
- /* Clump S/N values. */
- if(p->checkclumpsn)
- {
- p->clumpsn_s_name=gal_checkset_automatic_output(&p->cp, basename,
- ( p->cp.tableformat==GAL_TABLE_FORMAT_TXT
- ? "_clumpsn_sky.txt" : "_clumpsn_sky.fits") );
- p->clumpsn_d_name=gal_checkset_automatic_output(&p->cp, basename,
- ( p->cp.tableformat==GAL_TABLE_FORMAT_TXT
- ? "_clumpsn_det.txt" : "_clumpsn_det.fits") );
- }
-
- /* Segmentation steps. */
- if(p->checksegmentation)
- p->segmentationname=gal_checkset_automatic_output(&p->cp, basename,
- "_seg.fits");
}
@@ -410,58 +394,12 @@ ui_set_output_names(struct noisechiselparams *p)
static void
ui_prepare_kernel(struct noisechiselparams *p)
{
- /* The default kernel. It was created by saving the following commands in a
- script and running it. It will create a plain text array along with a
- FITS image. The crop is because the first and last rows and columns of
- all PSFs made by MakeProfiles are blank (zero) when you run with
- oversample=1. You can keep the spaces when copying and pasting ;-). Just
- make it executable and run it.
-
- set -o errexit # Stop if a program returns false.
- echo "0 0.0 0.0 3 2 0 0 1 1 5" > tmp.txt
- export GSL_RNG_TYPE=ranlxs2
- export GSL_RNG_SEED=1
- astmkprof tmp.txt --oversample=1 --envseed --numrandom=10000 \
- --tolerance=0.01 --nomerged
- astcrop 0_tmp.fits --section=2:*-1,2:*-1 --zeroisnotblank \
- --output=fwhm2.fits
- astconvertt fwhm2.fits --output=fwhm2.txt
- rm 0_tmp.fits tmp.txt
- */
- size_t kernel_2d_dsize[2]={11,11};
- float *f, *ff, *k, kernel_2d[121]=
- {
- 0, 0, 0, 0, 0, 6.57699e-09, 0, 0, 0, 0, 0,
-
- 0, 0, 6.57699e-09, 2.10464e-07, 1.68371e-06, 3.36742e-06, 1.68371e-06,
- 2.10464e-07, 6.57699e-09, 0, 0,
-
- 0, 6.57699e-09, 8.41855e-07, 2.69394e-05, 0.000383569, 0.000717224,
- 0.000379782, 2.69394e-05, 8.41855e-07, 6.57699e-09, 0,
-
- 0, 2.10464e-07, 2.69394e-05, 0.00140714, 0.00888549, 0.016448,
- 0.00867408, 0.00138203, 2.69394e-05, 2.10464e-07, 0,
-
- 0, 1.68371e-06, 0.000381138, 0.00875434, 0.0573377, 0.106308, 0.0570693,
- 0.00891745, 0.000378914, 1.68371e-06, 0,
-
- 6.57699e-09, 3.36742e-06, 0.00071364, 0.0164971, 0.106865, 0.197316,
- 0.106787, 0.0166434, 0.000713827, 3.36742e-06, 6.57699e-09,
-
- 0, 1.68371e-06, 0.000215515, 0.00894112, 0.0573699, 0.106239, 0.0567907,
- 0.00901191, 0.000215515, 1.68371e-06, 0,
-
- 0, 2.10464e-07, 2.69394e-05, 0.00135085, 0.0089288, 0.0164171,
- 0.00879334, 0.0013622, 2.69394e-05, 2.10464e-07, 0,
-
- 0, 6.57699e-09, 8.41855e-07, 2.69394e-05, 0.000215515, 0.000724137,
- 0.000215515, 2.69394e-05, 8.41855e-07, 6.57699e-09, 0,
-
- 0, 0, 6.57699e-09, 2.10464e-07, 1.68371e-06, 3.36742e-06, 1.68371e-06,
- 2.10464e-07, 6.57699e-09, 0, 0,
+ float *f, *ff, *k;
- 0, 0, 0, 0, 0, 6.57699e-09, 0, 0, 0, 0, 0
- };
+/* Since the default kernel has to be identical between NoiseChisel and
+ Segment, we have defined it in a shared header file to be accessible by
+ both programs. */
+#include <gnuastro-internal/kernel-2d.h>
/* If a kernel file is given, then use it. Otherwise, use the default
kernel. */
@@ -491,7 +429,7 @@ ui_prepare_kernel(struct noisechiselparams *p)
/* If a wide kernel is given, then read it into memory. Otherwise, just
ignore it. */
if(p->widekernelname)
- p->widekernel=gal_fits_img_read_kernel(p->widekernelname, p->wkhdu,
+ p->widekernel=gal_fits_img_read_kernel(p->widekernelname, p->whdu,
p->cp.minmapsize);
}
@@ -606,14 +544,14 @@ ui_preparations(struct noisechiselparams *p)
if(p->convolvedname)
{
/* Read the input convolved image. */
- p->conv = gal_fits_img_read_to_type(p->convolvedname, p->convolvedhdu,
+ p->conv = gal_fits_img_read_to_type(p->convolvedname, p->chdu,
GAL_TYPE_FLOAT32, p->cp.minmapsize);
/* Make sure the convolved image is the same size as the input. */
if( gal_data_dsize_is_different(p->input, p->conv) )
error(EXIT_FAILURE, 0, "%s (hdu %s), given to `--convolved' and "
"`--convolvehdu', is not the same size as NoiseChisel's "
- "input: %s (hdu: %s)", p->convolvedname, p->convolvedhdu,
+ "input: %s (hdu: %s)", p->convolvedname, p->chdu,
p->inputname, p->cp.hdu);
}
else
@@ -721,7 +659,7 @@ ui_read_check_inputs_setup(int argc, char *argv[],
printf(" - Input: %s (hdu: %s)\n", p->inputname, p->cp.hdu);
if(p->convolvedname)
printf(" - Convolved input: %s (hdu: %s)\n",
- p->convolvedname, p->convolvedhdu);
+ p->convolvedname, p->chdu);
else
{
if(p->kernelname)
@@ -739,7 +677,7 @@ ui_read_check_inputs_setup(int argc, char *argv[],
}
if(p->widekernelname)
printf(" - Wide Kernel: %s (hdu: %s)\n", p->widekernelname,
- p->wkhdu);
+ p->whdu);
}
}
@@ -820,9 +758,6 @@ ui_free_report(struct noisechiselparams *p, struct timeval
*t1)
if(p->detsn_s_name) free(p->detsn_s_name);
if(p->detsn_d_name) free(p->detsn_d_name);
if(p->detectionname) free(p->detectionname);
- if(p->clumpsn_s_name) free(p->clumpsn_s_name);
- if(p->clumpsn_d_name) free(p->clumpsn_d_name);
- if(p->segmentationname) free(p->segmentationname);
/* Free the allocated datasets. */
gal_data_free(p->sky);
@@ -832,7 +767,6 @@ ui_free_report(struct noisechiselparams *p, struct timeval
*t1)
gal_data_free(p->kernel);
gal_data_free(p->binary);
gal_data_free(p->olabel);
- gal_data_free(p->clabel);
gal_data_free(p->widekernel);
if(p->conv!=p->input) gal_data_free(p->conv);
diff --git a/bin/noisechisel/ui.h b/bin/noisechisel/ui.h
index 6e33e94..0722c38 100644
--- a/bin/noisechisel/ui.h
+++ b/bin/noisechisel/ui.h
@@ -1,5 +1,5 @@
/*********************************************************************
-NoiseChisel - Detect and segment signal in a noisy dataset.
+NoiseChisel - Detect signal in a noisy dataset.
NoiseChisel is part of GNU Astronomy Utilities (Gnuastro) package.
Original author:
@@ -50,8 +50,8 @@ enum program_args_groups
/* Available letters for short options:
- a b f j l n u x z
- A H J W X Y
+ a b f g i j l n u v x y z
+ A E G H J O W X Y
*/
enum option_keys_enum
{
@@ -59,7 +59,6 @@ enum option_keys_enum
UI_KEY_LARGETILESIZE = 'L',
UI_KEY_KERNEL = 'k',
UI_KEY_WIDEKERNEL = 'w',
- UI_KEY_SKYSUBTRACTED = 'E',
UI_KEY_MINSKYFRAC = 'B',
UI_KEY_MIRRORDIST = 'r',
UI_KEY_MODMEDQDIFF = 'Q',
@@ -68,15 +67,9 @@ enum option_keys_enum
UI_KEY_OPENING = 'p',
UI_KEY_SIGMACLIP = 's',
UI_KEY_DTHRESH = 'R',
- UI_KEY_DETSNMINAREA = 'i',
- UI_KEY_DETQUANT = 'c',
+ UI_KEY_SNMINAREA = 'm',
+ UI_KEY_SNQUANT = 'c',
UI_KEY_DETGROWQUANT = 'd',
- UI_KEY_SEGSNMINAREA = 'm',
- UI_KEY_SEGQUANT = 'g',
- UI_KEY_KEEPMAXNEARRIVER = 'v',
- UI_KEY_GTHRESH = 'G',
- UI_KEY_MINRIVERLENGTH = 'y',
- UI_KEY_OBJBORDERSN = 'O',
UI_KEY_CONTINUEAFTERCHECK = 'C',
@@ -84,11 +77,9 @@ enum option_keys_enum
automatically). */
UI_KEY_KHDU = 1000,
UI_KEY_CONVOLVED,
- UI_KEY_CONVOLVEDHDU,
- UI_KEY_WKHDU,
+ UI_KEY_CHDU,
+ UI_KEY_WHDU,
UI_KEY_MINNUMFALSE,
- UI_KEY_ONLYDETECTION,
- UI_KEY_GROWNCLUMPS,
UI_KEY_SMOOTHWIDTH,
UI_KEY_QTHRESHTILEQUANT,
UI_KEY_CHECKQTHRESH,
@@ -96,13 +87,12 @@ enum option_keys_enum
UI_KEY_NOERODEQUANT,
UI_KEY_OPENINGNGB,
UI_KEY_CHECKDETSKY,
- UI_KEY_CHECKDETSN,
+ UI_KEY_CHECKSN,
UI_KEY_DETGROWMAXHOLESIZE,
UI_KEY_CLEANGROWNDET,
UI_KEY_CHECKDETECTION,
UI_KEY_CHECKSKY,
- UI_KEY_CHECKCLUMPSN,
- UI_KEY_CHECKSEGMENTATION,
+ UI_KEY_RAWOUTPUT,
};
diff --git a/bin/TEMPLATE/Makefile.am b/bin/segment/Makefile.am
similarity index 85%
copy from bin/TEMPLATE/Makefile.am
copy to bin/segment/Makefile.am
index f784618..5bc28ca 100644
--- a/bin/TEMPLATE/Makefile.am
+++ b/bin/segment/Makefile.am
@@ -26,16 +26,16 @@ AM_CPPFLAGS = -I\$(top_srcdir)/bootstrapped/lib
## Program definition (name, linking, sources and headers)
-bin_PROGRAMS = astTEMPLATE
+bin_PROGRAMS = astsegment
-astTEMPLATE_LDADD = -lgnuastro
+astsegment_LDADD = -lgnuastro
-astTEMPLATE_SOURCES = main.c ui.c TEMPLATE.c
+astsegment_SOURCES = main.c ui.c segment.c clumps.c
-EXTRA_DIST = main.h authors-cite.h args.h ui.h TEMPLATE.h
+EXTRA_DIST = main.h authors-cite.h args.h ui.h segment.h clumps.h
## The configuration file (distribute and install).
## NOTE: the man page is created in doc/Makefile.am
-dist_sysconf_DATA = astTEMPLATE.conf
+dist_sysconf_DATA = astsegment.conf
diff --git a/bin/noisechisel/args.h b/bin/segment/args.h
similarity index 50%
copy from bin/noisechisel/args.h
copy to bin/segment/args.h
index 10a72a7..b7a66e5 100644
--- a/bin/noisechisel/args.h
+++ b/bin/segment/args.h
@@ -1,11 +1,11 @@
/*********************************************************************
-NoiseChisel - Detect and segment signal in a noisy dataset.
-NoiseChisel is part of GNU Astronomy Utilities (Gnuastro) package.
+Segment - Segment initial labels based on signal structure.
+Segment is part of GNU Astronomy Utilities (Gnuastro) package.
Original author:
Mohammad Akhlaghi <address@hidden>
Contributing author(s):
-Copyright (C) 2015-2018, Free Software Foundation, Inc.
+Copyright (C) 2018, Free Software Foundation, Inc.
Gnuastro is free software: you can redistribute it and/or modify it
under the terms of the GNU General Public License as published by the
@@ -28,159 +28,154 @@ along with Gnuastro. If not, see
<http://www.gnu.org/licenses/>.
-/* Options in argp_option format. */
+/* Array of acceptable options. */
struct argp_option program_options[] =
{
-
/* Input options. */
{
- "kernel",
- UI_KEY_KERNEL,
+ "detection",
+ UI_KEY_DETECTION,
"STR",
0,
- "Filename of kernel to convolve with input",
+ "Filename of detection image (to segment).",
GAL_OPTIONS_GROUP_INPUT,
- &p->kernelname,
+ &p->detectionname,
GAL_TYPE_STRING,
GAL_OPTIONS_RANGE_ANY,
GAL_OPTIONS_NOT_MANDATORY,
GAL_OPTIONS_NOT_SET
},
{
- "khdu",
- UI_KEY_KHDU,
+ "dhdu",
+ UI_KEY_DHDU,
"STR",
0,
- "HDU containing kernel image.",
+ "HDU containing detection image.",
GAL_OPTIONS_GROUP_INPUT,
- &p->khdu,
+ &p->dhdu,
GAL_TYPE_STRING,
GAL_OPTIONS_RANGE_ANY,
- GAL_OPTIONS_NOT_MANDATORY,
+ GAL_OPTIONS_MANDATORY,
GAL_OPTIONS_NOT_SET
},
{
- "convolved",
- UI_KEY_CONVOLVED,
+ "sky",
+ UI_KEY_SKY,
"STR",
0,
- "Convolved image file to avoid convolution.",
+ "Filename of Sky values image to subtract.",
GAL_OPTIONS_GROUP_INPUT,
- &p->convolvedname,
+ &p->skyname,
GAL_TYPE_STRING,
GAL_OPTIONS_RANGE_ANY,
GAL_OPTIONS_NOT_MANDATORY,
GAL_OPTIONS_NOT_SET
},
{
- "convolvedhdu",
- UI_KEY_CONVOLVEDHDU,
+ "skyhdu",
+ UI_KEY_SKYHDU,
"STR",
0,
- "HDU/extension of convolved image in file.",
+ "HDU containing Sky value to subtract.",
GAL_OPTIONS_GROUP_INPUT,
- &p->convolvedhdu,
+ &p->skyhdu,
GAL_TYPE_STRING,
GAL_OPTIONS_RANGE_ANY,
- GAL_OPTIONS_NOT_MANDATORY,
+ GAL_OPTIONS_MANDATORY,
GAL_OPTIONS_NOT_SET
},
{
- "widekernel",
- UI_KEY_WIDEKERNEL,
+ "std",
+ UI_KEY_STD,
"STR",
0,
- "Filename of wider kernel for better qthresh",
+ "Filename of Sky standard deviation.",
GAL_OPTIONS_GROUP_INPUT,
- &p->widekernelname,
+ &p->stdname,
GAL_TYPE_STRING,
GAL_OPTIONS_RANGE_ANY,
GAL_OPTIONS_NOT_MANDATORY,
GAL_OPTIONS_NOT_SET
},
{
- "wkhdu",
- UI_KEY_WKHDU,
+ "stdhdu",
+ UI_KEY_STDHDU,
"STR",
0,
- "HDU containing wide kernel image.",
+ "HDU containing Sky standard deviation.",
GAL_OPTIONS_GROUP_INPUT,
- &p->wkhdu,
+ &p->stdhdu,
GAL_TYPE_STRING,
GAL_OPTIONS_RANGE_ANY,
- GAL_OPTIONS_NOT_MANDATORY,
+ GAL_OPTIONS_MANDATORY,
GAL_OPTIONS_NOT_SET
},
{
- "skysubtracted",
- UI_KEY_SKYSUBTRACTED,
- 0,
+ "kernel",
+ UI_KEY_KERNEL,
+ "STR",
0,
- "Input is Sky subtracted (for error estimation).",
+ "Filename of kernel to convolve with input.",
GAL_OPTIONS_GROUP_INPUT,
- &p->skysubtracted,
- GAL_OPTIONS_NO_ARG_TYPE,
- GAL_OPTIONS_RANGE_0_OR_1,
+ &p->kernelname,
+ GAL_TYPE_STRING,
+ GAL_OPTIONS_RANGE_ANY,
GAL_OPTIONS_NOT_MANDATORY,
GAL_OPTIONS_NOT_SET
},
{
- "minskyfrac",
- UI_KEY_MINSKYFRAC,
- "FLT",
+ "khdu",
+ UI_KEY_KHDU,
+ "STR",
0,
- "Min. fraction of undetected area in tile.",
+ "HDU containing kernel image.",
GAL_OPTIONS_GROUP_INPUT,
- &p->minskyfrac,
- GAL_TYPE_FLOAT32,
- GAL_OPTIONS_RANGE_GE_0_LE_1,
- GAL_OPTIONS_MANDATORY,
+ &p->khdu,
+ GAL_TYPE_STRING,
+ GAL_OPTIONS_RANGE_ANY,
+ GAL_OPTIONS_NOT_MANDATORY,
GAL_OPTIONS_NOT_SET
},
{
- "minnumfalse",
- UI_KEY_MINNUMFALSE,
- "INT",
+ "convolved",
+ UI_KEY_CONVOLVED,
+ "STR",
0,
- "Minimum number for S/N estimation.",
+ "Convolved image file to avoid convolution.",
GAL_OPTIONS_GROUP_INPUT,
- &p->minnumfalse,
- GAL_TYPE_SIZE_T,
- GAL_OPTIONS_RANGE_GT_0,
- GAL_OPTIONS_MANDATORY,
+ &p->convolvedname,
+ GAL_TYPE_STRING,
+ GAL_OPTIONS_RANGE_ANY,
+ GAL_OPTIONS_NOT_MANDATORY,
GAL_OPTIONS_NOT_SET
},
-
-
- /* Tessellation. */
{
- "largetilesize",
- UI_KEY_LARGETILESIZE,
- "INT[,INT]",
+ "chdu",
+ UI_KEY_CHDU,
+ "STR",
0,
- "Sim. to --tilesize, but for larger tiles.",
- GAL_OPTIONS_GROUP_TESSELLATION,
- &p->ltl.tilesize,
- GAL_TYPE_SIZE_T,
- GAL_OPTIONS_RANGE_GT_0,
- GAL_OPTIONS_MANDATORY,
- GAL_OPTIONS_NOT_SET,
- gal_options_parse_sizes_reverse
+ "HDU/extension of convolved image in file.",
+ GAL_OPTIONS_GROUP_INPUT,
+ &p->chdu,
+ GAL_TYPE_STRING,
+ GAL_OPTIONS_RANGE_ANY,
+ GAL_OPTIONS_NOT_MANDATORY,
+ GAL_OPTIONS_NOT_SET
},
- /* Output options. */
+ /* Output. */
{
- "onlydetection",
- UI_KEY_ONLYDETECTION,
+ "rawoutput",
+ UI_KEY_RAWOUTPUT,
0,
0,
- "Stop at the end of detection.",
+ "Output only object and clump labels.",
GAL_OPTIONS_GROUP_OUTPUT,
- &p->onlydetection,
+ &p->rawoutput,
GAL_OPTIONS_NO_ARG_TYPE,
GAL_OPTIONS_RANGE_0_OR_1,
GAL_OPTIONS_NOT_MANDATORY,
@@ -191,344 +186,91 @@ struct argp_option program_options[] =
- /* Detection. */
- {
- 0, 0, 0, 0,
- "Detection:",
- UI_GROUP_DETECTION
- },
- {
- "mirrordist",
- UI_KEY_MIRRORDIST,
- "FLT",
- 0,
- "Max. dist. (error multip.) to find mode.",
- UI_GROUP_DETECTION,
- &p->mirrordist,
- GAL_TYPE_FLOAT32,
- GAL_OPTIONS_RANGE_ANY,
- GAL_OPTIONS_MANDATORY,
- GAL_OPTIONS_NOT_SET
- },
- {
- "modmedqdiff",
- UI_KEY_MODMEDQDIFF,
- "FLT",
- 0,
- "Max. mode and median quant diff. per tile.",
- UI_GROUP_DETECTION,
- &p->modmedqdiff,
- GAL_TYPE_FLOAT32,
- GAL_OPTIONS_RANGE_GE_0,
- GAL_OPTIONS_MANDATORY,
- GAL_OPTIONS_NOT_SET
- },
- {
- "qthresh",
- UI_KEY_QTHRESH,
- "FLT",
- 0,
- "Quantile threshold on convolved image.",
- UI_GROUP_DETECTION,
- &p->qthresh,
- GAL_TYPE_FLOAT32,
- GAL_OPTIONS_RANGE_GE_0_LT_1,
- GAL_OPTIONS_MANDATORY,
- GAL_OPTIONS_NOT_SET
- },
- {
- "qthreshtilequant",
- UI_KEY_QTHRESHTILEQUANT,
- "FLT",
- 0,
- "Remove tiles at higher quantiles.",
- UI_GROUP_DETECTION,
- &p->qthreshtilequant,
- GAL_TYPE_FLOAT32,
- GAL_OPTIONS_RANGE_GE_0_LE_1,
- GAL_OPTIONS_MANDATORY,
- GAL_OPTIONS_NOT_SET
- },
- {
- "smoothwidth",
- UI_KEY_SMOOTHWIDTH,
- "INT",
- 0,
- "Flat kernel width to smooth interpolated.",
- UI_GROUP_DETECTION,
- &p->smoothwidth,
- GAL_TYPE_SIZE_T,
- GAL_OPTIONS_RANGE_0_OR_ODD,
- GAL_OPTIONS_MANDATORY,
- GAL_OPTIONS_NOT_SET
- },
- {
- "checkqthresh",
- UI_KEY_CHECKQTHRESH,
- 0,
- 0,
- "Save quantile threshold estimation in file.",
- UI_GROUP_DETECTION,
- &p->checkqthresh,
- GAL_OPTIONS_NO_ARG_TYPE,
- GAL_OPTIONS_RANGE_0_OR_1,
- GAL_OPTIONS_NOT_MANDATORY,
- GAL_OPTIONS_NOT_SET
- },
- {
- "erode",
- UI_KEY_ERODE,
- "INT",
- 0,
- "Number of erosions after thresholding.",
- UI_GROUP_DETECTION,
- &p->erode,
- GAL_TYPE_SIZE_T,
- GAL_OPTIONS_RANGE_GE_0,
- GAL_OPTIONS_MANDATORY,
- GAL_OPTIONS_NOT_SET
- },
- {
- "erodengb",
- UI_KEY_ERODENGB,
- "INT",
- 0,
- "4 or 8 connectivity in erosion.",
- UI_GROUP_DETECTION,
- &p->erodengb,
- GAL_TYPE_SIZE_T,
- GAL_OPTIONS_RANGE_GT_0,
- GAL_OPTIONS_MANDATORY,
- GAL_OPTIONS_NOT_SET
- },
- {
- "noerodequant",
- UI_KEY_NOERODEQUANT,
- "FLT",
- 0,
- "Quantile for no erosion.",
- UI_GROUP_DETECTION,
- &p->noerodequant,
- GAL_TYPE_FLOAT32,
- GAL_OPTIONS_RANGE_GE_0_LE_1,
- GAL_OPTIONS_MANDATORY,
- GAL_OPTIONS_NOT_SET
- },
- {
- "opening",
- UI_KEY_OPENING,
- "INT",
- 0,
- "Depth of opening after erosion.",
- UI_GROUP_DETECTION,
- &p->opening,
- GAL_TYPE_SIZE_T,
- GAL_OPTIONS_RANGE_GT_0,
- GAL_OPTIONS_MANDATORY,
- GAL_OPTIONS_NOT_SET
- },
+ /* Tessellation. */
{
- "openingngb",
- UI_KEY_OPENINGNGB,
- "INT",
+ "largetilesize",
+ UI_KEY_LARGETILESIZE,
+ "INT[,INT]",
0,
- "4 or 8 connectivity in opening.",
- UI_GROUP_DETECTION,
- &p->openingngb,
+ "Sim. to --tilesize, but for larger tiles.",
+ GAL_OPTIONS_GROUP_TESSELLATION,
+ &p->ltl.tilesize,
GAL_TYPE_SIZE_T,
GAL_OPTIONS_RANGE_GT_0,
GAL_OPTIONS_MANDATORY,
- GAL_OPTIONS_NOT_SET
- },
- {
- "sigmaclip",
- UI_KEY_SIGMACLIP,
- "FLT,FLT",
- 0,
- "Sigma multiple and, tolerance or number.",
- UI_GROUP_DETECTION,
- &p->sigmaclip,
- GAL_TYPE_STRING,
- GAL_OPTIONS_RANGE_ANY,
- GAL_OPTIONS_MANDATORY,
GAL_OPTIONS_NOT_SET,
- gal_options_read_sigma_clip
- },
- {
- "checkdetsky",
- UI_KEY_CHECKDETSKY,
- 0,
- 0,
- "Save Sky value estimation for pseudo-dets.",
- UI_GROUP_DETECTION,
- &p->checkdetsky,
- GAL_OPTIONS_NO_ARG_TYPE,
- GAL_OPTIONS_RANGE_0_OR_1,
- GAL_OPTIONS_NOT_MANDATORY,
- GAL_OPTIONS_NOT_SET
- },
- {
- "dthresh",
- UI_KEY_DTHRESH,
- "FLT",
- 0,
- "Sigma threshold for Pseudo-detections.",
- UI_GROUP_DETECTION,
- &p->dthresh,
- GAL_TYPE_FLOAT32,
- GAL_OPTIONS_RANGE_ANY,
- GAL_OPTIONS_MANDATORY,
- GAL_OPTIONS_NOT_SET
- },
- {
- "detsnminarea",
- UI_KEY_DETSNMINAREA,
- "INT",
- 0,
- "Min. pseudo-detection area for S/N dist.",
- UI_GROUP_DETECTION,
- &p->detsnminarea,
- GAL_TYPE_SIZE_T,
- GAL_OPTIONS_RANGE_ANY,
- GAL_OPTIONS_MANDATORY,
- GAL_OPTIONS_NOT_SET
- },
- {
- "checkdetsn",
- UI_KEY_CHECKDETSN,
- 0,
- 0,
- "Save pseudo-detection S/N values to a file.",
- UI_GROUP_DETECTION,
- &p->checkdetsn,
- GAL_OPTIONS_NO_ARG_TYPE,
- GAL_OPTIONS_RANGE_0_OR_1,
- GAL_OPTIONS_NOT_MANDATORY,
- GAL_OPTIONS_NOT_SET
+ gal_options_parse_sizes_reverse
},
+
+
+
+
+ /* Segmentation options. */
{
- "detquant",
- UI_KEY_DETQUANT,
- "FLT",
- 0,
- "Quantile in pseudo-det. to define true.",
- UI_GROUP_DETECTION,
- &p->detquant,
- GAL_TYPE_FLOAT32,
- GAL_OPTIONS_RANGE_GT_0_LT_1,
- GAL_OPTIONS_MANDATORY,
- GAL_OPTIONS_NOT_SET
+ 0, 0, 0, 0,
+ "Segmentation:",
+ UI_GROUP_SEGMENTATION
},
{
- "detgrowquant",
- UI_KEY_DETGROWQUANT,
+ "minskyfrac",
+ UI_KEY_MINSKYFRAC,
"FLT",
0,
- "Minimum quant. to expand true detections.",
- UI_GROUP_DETECTION,
- &p->detgrowquant,
+ "Min. fraction of undetected area in tile.",
+ UI_GROUP_SEGMENTATION,
+ &p->minskyfrac,
GAL_TYPE_FLOAT32,
GAL_OPTIONS_RANGE_GE_0_LE_1,
GAL_OPTIONS_MANDATORY,
GAL_OPTIONS_NOT_SET
},
{
- "detgrowmaxholesize",
- UI_KEY_DETGROWMAXHOLESIZE,
+ "snminarea",
+ UI_KEY_SNMINAREA,
"INT",
0,
- "Max. area of holes after growth to fill.",
- UI_GROUP_DETECTION,
- &p->detgrowmaxholesize,
+ "Minimum area of clumps for S/N estimation.",
+ UI_GROUP_SEGMENTATION,
+ &p->snminarea,
GAL_TYPE_SIZE_T,
- GAL_OPTIONS_RANGE_GE_0,
+ GAL_OPTIONS_RANGE_GT_0,
GAL_OPTIONS_MANDATORY,
GAL_OPTIONS_NOT_SET
},
{
- "cleangrowndet",
- UI_KEY_CLEANGROWNDET,
- 0,
- 0,
- "Remove small S/N grown detections.",
- UI_GROUP_DETECTION,
- &p->cleangrowndet,
- GAL_OPTIONS_NO_ARG_TYPE,
- GAL_OPTIONS_RANGE_ANY,
- GAL_OPTIONS_NOT_MANDATORY,
- GAL_OPTIONS_NOT_SET
- },
- {
- "checkdetection",
- UI_KEY_CHECKDETECTION,
- 0,
- 0,
- "Save all the detection steps to a file.",
- UI_GROUP_DETECTION,
- &p->checkdetection,
- GAL_OPTIONS_NO_ARG_TYPE,
- GAL_OPTIONS_RANGE_0_OR_1,
- GAL_OPTIONS_NOT_MANDATORY,
- GAL_OPTIONS_NOT_SET
- },
- {
- "checksky",
- UI_KEY_CHECKSKY,
+ "checksn",
+ UI_KEY_CHECKSN,
0,
0,
- "Final sky and its STD steps in a file.",
- UI_GROUP_DETECTION,
- &p->checksky,
+ "Save clump S/N values into a file.",
+ UI_GROUP_SEGMENTATION,
+ &p->checksn,
GAL_OPTIONS_NO_ARG_TYPE,
GAL_OPTIONS_RANGE_0_OR_1,
GAL_OPTIONS_NOT_MANDATORY,
GAL_OPTIONS_NOT_SET
},
-
-
-
-
-
- /* Segmentation */
- {
- 0, 0, 0, 0,
- "Segmentation:",
- UI_GROUP_SEGMENTATION
- },
{
- "segsnminarea",
- UI_KEY_SEGSNMINAREA,
+ "minnumfalse",
+ UI_KEY_MINNUMFALSE,
"INT",
0,
- "Minimum area of clumps for S/N estimation.",
+ "Minimum number for S/N estimation.",
UI_GROUP_SEGMENTATION,
- &p->segsnminarea,
+ &p->minnumfalse,
GAL_TYPE_SIZE_T,
GAL_OPTIONS_RANGE_GT_0,
GAL_OPTIONS_MANDATORY,
GAL_OPTIONS_NOT_SET
},
{
- "checkclumpsn",
- UI_KEY_CHECKCLUMPSN,
- 0,
- 0,
- "Save Sky clump S/N values into a file.",
- UI_GROUP_SEGMENTATION,
- &p->checkclumpsn,
- GAL_OPTIONS_NO_ARG_TYPE,
- GAL_OPTIONS_RANGE_0_OR_1,
- GAL_OPTIONS_NOT_MANDATORY,
- GAL_OPTIONS_NOT_SET
- },
- {
- "segquant",
- UI_KEY_SEGQUANT,
+ "snquant",
+ UI_KEY_SNQUANT,
"FLT",
0,
"S/N Quantile of true sky clumps.",
UI_GROUP_SEGMENTATION,
- &p->segquant,
+ &p->snquant,
GAL_TYPE_FLOAT32,
GAL_OPTIONS_RANGE_GT_0,
GAL_OPTIONS_MANDATORY,
@@ -548,6 +290,19 @@ struct argp_option program_options[] =
GAL_OPTIONS_NOT_SET
},
{
+ "clumpsnthresh",
+ UI_KEY_CLUMPSNTHRESH,
+ "FLT",
+ 0,
+ "S/N threshold of true clumps.",
+ UI_GROUP_SEGMENTATION,
+ &p->clumpsnthresh,
+ GAL_TYPE_FLOAT32,
+ GAL_OPTIONS_RANGE_GT_0,
+ GAL_OPTIONS_NOT_MANDATORY,
+ GAL_OPTIONS_NOT_SET
+ },
+ {
"gthresh",
UI_KEY_GTHRESH,
"FLT",
@@ -636,7 +391,6 @@ struct argp_option program_options[] =
- /* End of options. */
{0}
};
@@ -644,7 +398,10 @@ struct argp_option program_options[] =
-/* Define the child argp structure. */
+/* Define the child argp structure
+ -------------------------------
+
+ NOTE: these parts can be left untouched.*/
struct argp
gal_options_common_child = {gal_commonopts_options,
gal_options_common_argp_parse,
diff --git a/bin/segment/astsegment.conf b/bin/segment/astsegment.conf
new file mode 100644
index 0000000..e5a3b7f
--- /dev/null
+++ b/bin/segment/astsegment.conf
@@ -0,0 +1,41 @@
+# Default parameters (System) for Segment.
+# Segment is part of GNU Astronomy Utilities.
+#
+# Use the long option name of each parameter followed by a value. The name
+# and value should be separated by atleast one white-space character (for
+# example ` '[space], or tab). Lines starting with `#' are ignored.
+#
+# For more information, please run these commands:
+#
+# $ astsegment --help # Full list of options, short doc.
+# $ astsegment -P # Print all options and used values.
+# $ info astsegment # All options and input/output.
+# $ info gnuastro "Configuration files" # How to use configuration files.
+#
+# Copying and distribution of this file, with or without modification, are
+# permitted in any medium without royalty provided the copyright notice and
+# this notice are preserved. This file is offered as-is, without any
+# warranty.
+
+# Input:
+ khdu 1
+ chdu 1
+ dhdu DETECTIONS
+ skyhdu SKY
+ stdhdu SKY_STD
+ minskyfrac 0.7
+ minnumfalse 100
+
+# Tessellation
+ largetilesize 200,200
+
+# Segmentation
+ snminarea 15
+ keepmaxnearriver 0
+ snquant 0.95
+ gthresh 0.5
+ minriverlength 15
+ objbordersn 1
+
+# Operating mode
+ continueaftercheck 0
diff --git a/bin/noisechisel/authors-cite.h b/bin/segment/authors-cite.h
similarity index 88%
copy from bin/noisechisel/authors-cite.h
copy to bin/segment/authors-cite.h
index 782d42d..8b6855b 100644
--- a/bin/noisechisel/authors-cite.h
+++ b/bin/segment/authors-cite.h
@@ -1,11 +1,11 @@
/*********************************************************************
-NoiseChisel - Detect and segment signal in a noisy dataset.
-NoiseChisel is part of GNU Astronomy Utilities (Gnuastro) package.
+Segment - Segment initial labels based on signal structure.
+Segment is part of GNU Astronomy Utilities (Gnuastro) package.
Original author:
Mohammad Akhlaghi <address@hidden>
Contributing author(s):
-Copyright (C) 2017-2018, Free Software Foundation, Inc.
+Copyright (C) 2018, Free Software Foundation, Inc.
Gnuastro is free software: you can redistribute it and/or modify it
under the terms of the GNU General Public License as published by the
diff --git a/bin/noisechisel/clumps.c b/bin/segment/clumps.c
similarity index 60%
rename from bin/noisechisel/clumps.c
rename to bin/segment/clumps.c
index 76d62cc..31a138f 100644
--- a/bin/noisechisel/clumps.c
+++ b/bin/segment/clumps.c
@@ -1,6 +1,6 @@
/*********************************************************************
-NoiseChisel - Detect and segment signal in a noisy dataset.
-NoiseChisel is part of GNU Astronomy Utilities (Gnuastro) package.
+Segment - Segment initial labels based on signal structure.
+Segment is part of GNU Astronomy Utilities (Gnuastro) package.
Original author:
Mohammad Akhlaghi <address@hidden>
@@ -31,6 +31,7 @@ along with Gnuastro. If not, see
<http://www.gnu.org/licenses/>.
#include <gnuastro/fits.h>
#include <gnuastro/qsort.h>
#include <gnuastro/blank.h>
+#include <gnuastro/label.h>
#include <gnuastro/threads.h>
#include <gnuastro/dimension.h>
#include <gnuastro/statistics.h>
@@ -42,353 +43,6 @@ along with Gnuastro. If not, see
<http://www.gnu.org/licenses/>.
#include "ui.h"
#include "clumps.h"
-#include "threshold.h"
-
-
-
-
-
-
-
-
-
-
-/****************************************************************
- ***************** Over segmentation ********************
- ****************************************************************/
-/* Over-segment the region specified by its indexs into peaks and their
- respective regions (clumps). This is very similar to the immersion
- method of Vincent & Soille(1991), but here, we will not separate the
- image into layers, instead, we will work based on the ordered flux
- values. If a certain pixel (at a certain level) has no neighbors, it is
- a local maximum and will be assigned a new label. If it has a labeled
- neighbor, it will take that label and if there is more than one
- neighboring labeled region that pixel will be a `river` pixel. */
-void
-clumps_oversegment(struct clumps_thread_params *cltprm)
-{
- struct noisechiselparams *p=cltprm->clprm->p;
- size_t ndim=p->input->ndim;
-
- float *arr=p->conv->array;
- gal_data_t *indexs=cltprm->indexs;
- gal_list_sizet_t *Q=NULL, *cleanup=NULL;
- size_t *a, *af, ind, *dsize=p->input->dsize;
- size_t *dinc=gal_dimension_increment(ndim, dsize);
- int32_t n1, nlab, rlab, curlab=1, *clabel=p->clabel->array;
-
- /*********************************************
- For checks and debugging:*
- gal_data_t *crop;
- size_t extcount=1;
- int32_t *cr, *crf;
- size_t checkdsize[2]={10,10};
- size_t checkstart[2]={50,145};
- char *filename="clumpbuild.fits";
- size_t checkstartind=gal_dimension_coord_to_index(2, dsize, checkstart);
- gal_data_t *tile=gal_data_alloc(gal_data_ptr_increment(arr, checkstartind,
- p->conv->type),
- GAL_TYPE_INVALID, 2, checkdsize,
- NULL, 0, 0, NULL, NULL, NULL);
- tile->block=p->conv;
- gal_checkset_writable_remove(filename, 0, 0);
- if(p->cp.numthreads!=1)
- error(EXIT_FAILURE, 0, "in the debugging mode of `clumps_oversegment' "
- "only one thread must be used");
- crop=gal_data_copy(tile);
- gal_fits_img_write(crop, filename, NULL, PROGRAM_NAME);
- gal_data_free(crop);
- printf("blank: %u\nriver: %u\ntmpcheck: %u\ninit: %u\nmaxlab: %u\n",
- (int32_t)GAL_BLANK_INT32, (int32_t)CLUMPS_RIVER,
- (int32_t)CLUMPS_TMPCHECK, (int32_t)CLUMPS_INIT,
- (int32_t)CLUMPS_MAXLAB);
- tile->array=gal_tile_block_relative_to_other(tile, p->clabel);
- tile->block=p->clabel;
- **********************************************/
-
-
- /* If the size of the indexs is zero, then this function is pointless. */
- if(indexs->size==0) { cltprm->numinitclumps=0; return; }
-
-
- /* Sort the given indexs based on their flux (`gal_qsort_index_arr' is
- defined as static in `gnuastro/qsort.h') */
- gal_qsort_index_arr=p->conv->array;
- qsort(indexs->array, indexs->size, sizeof(size_t),
- gal_qsort_index_float_decreasing);
-
-
- /* Initialize the region we want to over-segment. */
- af=(a=indexs->array)+indexs->size; do clabel[*a]=CLUMPS_INIT; while(++a<af);
-
-
- /* Go over all the given indexs and pull out the clumps. */
- af=(a=indexs->array)+indexs->size;
- do
- /* When regions of a constant flux or masked regions exist, some later
- indexs (although they have same flux) will be filled before hand. If
- they are done, there is no need to do them again. */
- if(clabel[*a]==CLUMPS_INIT)
- {
- /* It might happen where one or multiple regions of the pixels
- under study have the same flux. So two equal valued pixels of
- two separate (but equal flux) regions will fall immediately
- after each other in the sorted list of indexs and we have to
- account for this.
-
- Therefore, if we see that the next pixel in the index list has
- the same flux as this one, it does not guarantee that it should
- be given the same label. Similar to the breadth first search
- algorithm for finding connected components, we will search all
- the neighbours and the neighbours of those neighbours that have
- the same flux of this pixel to see if they touch any label or
- not and to finally give them all the same label. */
- if( (a+1)<af && arr[*a]==arr[*(a+1)] )
- {
- /* Label of first neighbor found. */
- n1=0;
-
- /* A small sanity check. */
- if(Q!=NULL || cleanup!=NULL)
- error(EXIT_FAILURE, 0, "%s: a bug! Please contact us at %s so "
- "we can fix this problem. `Q' and `cleanup' should be "
- "NULL but while checking the equal flux regions they "
- "aren't", __func__, PACKAGE_BUGREPORT);
-
- /* Add this pixel to a queue. */
- gal_list_sizet_add(&Q, *a);
- gal_list_sizet_add(&cleanup, *a);
- clabel[*a] = CLUMPS_TMPCHECK;
-
- /* Find all the pixels that have the same flux and are
- connected. */
- while(Q!=NULL)
- {
- /* Pop an element from the queue. */
- ind=gal_list_sizet_pop(&Q);
-
- /* Look at the neighbors and see if we already have a
- label. */
- GAL_DIMENSION_NEIGHBOR_OP(ind, ndim, dsize, ndim, dinc,
- {
- /* If it is already decided to be a river, then stop
- looking at the neighbors. */
- if(n1!=CLUMPS_RIVER)
- {
- /* For easy reading. */
- nlab=clabel[ nind ];
-
- /* This neighbor's label isn't zero. */
- if(nlab)
- {
- /* If this neighbor has not been labeled yet
- and has an equal flux, add it to the queue
- to expand the studied region.*/
- if( nlab==CLUMPS_INIT && arr[nind]==arr[*a] )
- {
- clabel[nind]=CLUMPS_TMPCHECK;
- gal_list_sizet_add(&Q, nind);
- gal_list_sizet_add(&cleanup, nind);
- }
- else
- n1=( nlab>0
-
- /* If this neighbor has a positive
- nlab, it belongs to another object,
- so if `n1' has not been set for the
- whole region (n1==0), put `nlab'
- into `n1'. If `n1' has been set and
- is different from `nlab' then this
- whole equal flux region should be a
- wide river because it is connecting
- two connected regions.*/
- ? ( n1
- ? (n1==nlab ? n1 : CLUMPS_RIVER)
- : nlab )
-
- /* If the data has blank pixels (recall
- that blank in int32 is negative),
- see if the neighbor is blank and if
- so, set the label to a river. Since
- the flag checking can be done
- outside this loop, for datasets with
- no blank element this last step will
- be completley ignored. */
- : ( ( (p->input->flag
- & GAL_DATA_FLAG_HASBLANK)
- && nlab==GAL_BLANK_INT32 )
- ? CLUMPS_RIVER : n1 ) );
- }
-
- /* If this neigbour has a label of zero, then we
- are on the edge of the indexed region (the
- neighbor is not in the initial list of pixels
- to segment). When over-segmenting the noise and
- the detections, `clabel' is zero for the parts
- of the image that we are not interested in
- here. */
- else clabel[*a]=CLUMPS_RIVER;
- }
- } );
- }
-
- /* Set the label that is to be given to this equal flux
- region. If `n1' was set to any value, then that label should
- be used for the whole region. Otherwise, this is a new
- label, see the case for a non-flat region. */
- if(n1) rlab = n1;
- else
- {
- rlab = curlab++;
- if( cltprm->topinds ) /* This is a local maximum of */
- cltprm->topinds[rlab]=*a; /* this region, save its index.*/
- }
-
- /* Give the same label to the whole connected equal flux
- region, except those that might have been on the side of
- the image and were a river pixel. */
- while(cleanup!=NULL)
- {
- ind=gal_list_sizet_pop(&cleanup);
- /* If it was on the sides of the image, it has been
- changed to a river pixel. */
- if( clabel[ ind ]==CLUMPS_TMPCHECK ) clabel[ ind ]=rlab;
- }
- }
-
- /* The flux of this pixel is not the same as the next sorted
- flux, so simply find the label for this object. */
- else
- {
- /* `n1' is the label of the first labeled neighbor found, so
- we'll initialize it to zero. */
- n1=0;
-
- /* Go over all the fully connected neighbors of this pixel and
- see if all the neighbors (with maximum connectivity: the
- number of dimensions) that have a non-macro value (less than
- CLUMPS_MAXLAB) belong to one label or not. If the pixel is
- neighboured by more than one label, set it as a river
- pixel. Also if it is touching a zero valued pixel (which
- does not belong to this object), set it as a river pixel.*/
- GAL_DIMENSION_NEIGHBOR_OP(*a, ndim, dsize, ndim, dinc,
- {
- /* When `n1' has already been set as a river, there is no
- point in looking at the other neighbors. */
- if(n1!=CLUMPS_RIVER)
- {
- /* For easy reading. */
- nlab=clabel[ nind ];
-
- /* If this neighbor is on a non-processing label, then
- set the first neighbor accordingly. Note that we
- also want the zero valued neighbors (detections if
- working on sky, and sky if working on detection):
- we want rivers between the two domains. */
- n1 = ( nlab
-
- /* nlab is non-zero. */
- ? ( nlab>0
-
- /* Neighbor has a meaningful label, so
- check with any previously found labeled
- neighbors. */
- ? ( n1
- ? ( nlab==n1 ? n1 : CLUMPS_RIVER )
- : nlab )
-
- /* If the dataset has blank values and this
- neighbor is blank, then the pixel should
- be a river. Note that the blank checking
- can be optimized out, so if the input
- doesn't have blank values,
- `nlab==GAL_BLANK_INT32' will never be
- checked. */
- : ( (p->input->flag & GAL_DATA_FLAG_HASBLANK)
- && nlab==GAL_BLANK_INT32
- ? CLUMPS_RIVER : n1 ) )
-
- /* `nlab==0' (the neighbor lies in the other
- domain (sky or detections). To avoid the
- different domains touching, this pixel
- should be a river. */
- : CLUMPS_RIVER );
- }
- });
-
- /* Either assign a new label to this pixel, or give it the one
- of its neighbors. If n1 equals zero, then this is a new
- peak, and a new label should be created. But if n1!=0, it
- is either a river pixel (has more than one labeled neighbor
- and has been set to `CLUMPS_RIVER' before) or all its
- neighbors have the same label. In both such cases, rlab
- should be set to n1.*/
- if(n1) rlab = n1;
- else
- {
- rlab = curlab++;
- if( cltprm->topinds )
- cltprm->topinds[ rlab ]=*a;
- }
-
- /* Put the found label in the pixel. */
- clabel[ *a ] = rlab;
- }
-
- /*********************************************
- For checks and debugging:
- if( *a / dsize[1] >= checkstart[0]
- && *a / dsize[1] < checkstart[0] + checkdsize[0]
- && *a % dsize[1] >= checkstart[1]
- && *a % dsize[1] < checkstart[1] + checkdsize[1] )
- {
- printf("%zu (%zu: %zu, %zu): %u\n", ++extcount, *a,
- (*a%dsize[1])-checkstart[1], (*a/dsize[1])-checkstart[0],
- clabel[*a]);
- crop=gal_data_copy(tile);
- crf=(cr=crop->array)+crop->size;
- do if(*cr==CLUMPS_RIVER) *cr=0; while(++cr<crf);
- gal_fits_img_write(crop, filename, NULL, PROGRAM_NAME);
- gal_data_free(crop);
- }
- **********************************************/
- }
- while(++a<af);
-
- /* Save the total number of clumps. */
- cltprm->numinitclumps=curlab-1;
-
- /* Set all the river pixels to zero. Note that this is only necessary for
- the detected clumps. When finding clumps over the Sky, we will be
- going over the full tile and removing rivers after this function. This
- is because, we set the borders of the tile to a river value and didn't
- include them in the list of indexs. */
- if(cltprm->clprm->sky0_det1)
- {
- af=(a=indexs->array)+indexs->size;
- do if( clabel[*a]==CLUMPS_RIVER ) clabel[*a]=CLUMPS_INIT; while(++a<af);
- }
-
- /*********************************************
- For checks and debugging:
- tile->array=NULL;
- gal_data_free(tile);
- printf("Total number of clumps: %u\n", curlab-1);
- **********************************************/
-
- /* Clean up. */
- free(dinc);
-}
-
-
-
-
-
-
-
-
-
-
@@ -411,11 +65,11 @@ clumps_grow_prepare_initial(struct clumps_thread_params
*cltprm)
{
gal_data_t *indexs=cltprm->indexs;
gal_data_t *input=cltprm->clprm->p->input;
- struct noisechiselparams *p=cltprm->clprm->p;
+ struct segmentparams *p=cltprm->clprm->p;
- size_t *s, *sf, *dsize=input->dsize;
size_t ndiffuse=0, coord[2], *dindexs;
- double wcoord[2]={0.0f,0.0f}, brightness=0.0f;
+ double wcoord[2]={0.0f,0.0f}, sum=0.0f;
+ size_t *s, *sf, *dsize=input->dsize, ndim=input->ndim;
float glimit, *imgss=input->array, *std=p->std->array;
int32_t *olabel=p->olabel->array, *clabel=p->clabel->array;
@@ -426,7 +80,7 @@ clumps_grow_prepare_initial(struct clumps_thread_params
*cltprm)
do
if( imgss[ *s ] > 0.0f )
{
- brightness += imgss[ *s ];
+ sum += imgss[ *s ];
wcoord[0] += imgss[ *s ] * (*s/dsize[1]);
wcoord[1] += imgss[ *s ] * (*s%dsize[1]);
}
@@ -435,7 +89,7 @@ clumps_grow_prepare_initial(struct clumps_thread_params
*cltprm)
/* Calculate the center, if no pixels were positive, use the
geometric center (irrespective of flux). */
- if(brightness==0.0f)
+ if(sum==0.0f)
{
sf=(s=indexs->array)+indexs->size;
do
@@ -444,17 +98,24 @@ clumps_grow_prepare_initial(struct clumps_thread_params
*cltprm)
wcoord[1] += *s % dsize[1];
}
while(++s<sf);
- brightness = indexs->size;
+ sum = indexs->size;
}
- /* Convert floatint point coordinates to FITS integers. */
- coord[0] = GAL_DIMENSION_FLT_TO_INT(wcoord[0]);
- coord[1] = GAL_DIMENSION_FLT_TO_INT(wcoord[1]);
+ /* Convert floating point coordinates to integers. */
+ coord[0] = GAL_DIMENSION_FLT_TO_INT(wcoord[0]/sum);
+ coord[1] = GAL_DIMENSION_FLT_TO_INT(wcoord[1]/sum);
+
+ /* Find the growth limit. Note that the STD image may be a full sized
+ image or a tessellation. We'll check through the number of elements it
+ has. */
+ cltprm->std = ( p->std->size==p->input->size
+ ? std[ gal_dimension_coord_to_index(ndim, dsize, coord) ]
+ : std[ gal_tile_full_id_from_coord(&p->cp.tl, coord) ] );
- /* Find the growth limit */
- cltprm->std = std[ gal_tile_full_id_from_coord(&p->cp.tl, coord) ];
+
+ /* From the standard deviation, find the growth limit. */
glimit = p->gthresh * cltprm->std;
@@ -473,7 +134,7 @@ clumps_grow_prepare_initial(struct clumps_thread_params
*cltprm)
do
{
olabel[*s] = clabel[*s];
- if( clabel[*s]==CLUMPS_INIT )
+ if( clabel[*s]==GAL_LABEL_INIT )
if( imgss[*s]>glimit ) dindexs[ ndiffuse++ ] = *s;
}
while(++s<sf);
@@ -521,145 +182,7 @@ clumps_grow_prepare_final(struct clumps_thread_params
*cltprm)
-/* Grow the true clumps over the diffuse regions of a detection. Note that
- unlike before, were river pixels would get a separate label for them
- selves, here, they don't, they just get set back to SEGMENTINIT. This is
- because some of the pixels that lie immediately between two labeled
- regions might not be in the blankinds array (they were below the
- threshold). So we have to find river pixels later on, after the growth
- is done independently.
- This function is going to be used before identifying objects and also
- after it (to completely fill in the diffuse area). The distinguishing
- point between these two steps is the presence of rivers, so you can use
- the `withrivers' argument.
-
-
- Input:
-
- labels: The labels array that must be operated on. The pixels that
- must be "grown" must have the value `CLUMPS_INIT' (negative).
-
- diffuseindexs: The indexs of the pixels that must be grown.
-
- withrivers: as described above.
-
- connectivity: connectivity to define neighbors for growth.
-*/
-void
-clumps_grow(gal_data_t *labels, gal_data_t *diffuseindexs, int withrivers,
- int connectivity)
-{
- int searchngb;
- size_t *diarray=diffuseindexs->array;
- int32_t n1, nlab, *olabel=labels->array;
- size_t *s, *sf, thisround, ndiffuse=diffuseindexs->size;
- size_t *dinc=gal_dimension_increment(labels->ndim, labels->dsize);
-
- /* A small sanity check: */
- if(labels->type!=GAL_TYPE_INT32)
- error(EXIT_FAILURE, 0, "%s: `labels' has to have type of int32_t",
- __func__);
- if(diffuseindexs->type!=GAL_TYPE_SIZE_T)
- error(EXIT_FAILURE, 0, "%s: `diffuseindexs' has to have type of size_t",
- __func__);
-
- /* The basic idea is this: after growing, not all the blank pixels are
- necessarily filled, for example the pixels might belong to two regions
- above the growth threshold. So the pixels in between them (which are
- below the threshold will not ever be able to get a label). Therefore,
- the safest way we can terminate the loop of growing the objects is to
- stop it when the number of pixels left to fill in this round
- (thisround) equals the number of blanks.
-
- To start the loop, we set `thisround' to one more than the number of
- diffuse pixels. Note that it will be corrected immediately after the
- loop has started, it is just important to pass the `while'. */
- thisround=ndiffuse+1;
- while( thisround > ndiffuse )
- {
- /* `thisround' will keep the number of pixels to be inspected in this
- round. `ndiffuse' will count the number of pixels left without a
- label by the end of this round. Since `ndiffuse' comes from the
- previous loop (or outside, for the first round) it has to be saved
- in `thisround' to begin counting a fresh. */
- thisround=ndiffuse;
- ndiffuse=0;
-
- /* Go over all the available indexs. */
- sf=(s=diffuseindexs->array)+diffuseindexs->size;
- do
- {
- /* We'll begin by assuming the nearest neighbor of this pixel has
- no label (has a value of 0). */
- n1=0;
-
- /* Check the very closest neighbors of each pixel (4-connectivity
- in a 2D image). Note that since this macro has multiple loops
- within it, we can't use break. We'll use a variable instead. */
- searchngb=1;
- GAL_DIMENSION_NEIGHBOR_OP(*s, labels->ndim, labels->dsize,
- connectivity, dinc,
- {
- if(searchngb)
- {
- /* For easy reading. */
- nlab = olabel[nind];
-
- /* This neighbor's label is meaningful. */
- if(nlab>0) /* This is a real label. */
- {
- if(n1) /* A prev. neighboring label has been found. */
- {
- if( n1 != nlab ) /* Different label from */
- { /* prevously found neighbor for this pixel. */
- n1=CLUMPS_RIVER;
- searchngb=0;
- }
- }
- else
- { /* This is the first labeld neighbor found. */
- n1=nlab;
-
- /* If we want to completely fill in the region
- (`withrivers==0'), then there is no point in
- looking in other neighbors, the first neighbor
- we find is the one we'll use. */
- if(!withrivers) searchngb=0;
- }
- }
- }
- } );
-
- /* The loop above over neighbors finishes with three
- possibilities:
-
- n1==0 --> No labeled neighbor was found.
- n1==CLUMPS_RIVER --> Connecting two labeled regions.
- n1>0 --> Only has one neighbouring label.
-
- The first one means that no neighbors were found and this
- pixel should be kept for the next loop (we'll be growing the
- objects pixel-layer by pixel-layer). In the other two cases,
- we just need to write in the value of `n1'. */
- if(n1)
- {
- olabel[*s]=n1;
- if(withrivers && n1==CLUMPS_RIVER) /* To keep rivers in */
- diarray[ ndiffuse++ ] = *s; /* the diffuse list. */
- }
- else
- diarray[ ndiffuse++ ] = *s;
-
- /* Correct the size of the `diffuseindexs' dataset. */
- diffuseindexs->size = diffuseindexs->dsize[0] = ndiffuse;
- }
- while(++s<sf);
- }
-
- /* Clean up. */
- free(dinc);
-}
@@ -705,12 +228,13 @@ enum infocols
static void
clumps_get_raw_info(struct clumps_thread_params *cltprm)
{
- struct noisechiselparams *p=cltprm->clprm->p;
+ struct segmentparams *p=cltprm->clprm->p;
size_t ndim=p->input->ndim, *dsize=p->input->dsize;
size_t i, *a, *af, ii, coord[2];
double *row, *info=cltprm->info->array;
size_t nngb=gal_dimension_num_neighbors(ndim);
+ struct gal_tile_two_layer_params *tl=&p->cp.tl;
float *arr=p->input->array, *std=p->std->array;
size_t *dinc=gal_dimension_increment(ndim, dsize);
int32_t lab, nlab, *ngblabs, *clabel=p->clabel->array;
@@ -786,7 +310,7 @@ clumps_get_raw_info(struct clumps_thread_params *cltprm)
for(lab=1; lab<=cltprm->numinitclumps; ++lab)
{
row = &info [ lab * INFO_NCOLS ];
- if ( row[INFO_INAREA] > p->segsnminarea )
+ if ( row[INFO_INAREA] > p->snminarea )
{
/* Especially over the undetected regions, it might happen that
none of the pixels were positive. In that case, set the total
@@ -796,8 +320,12 @@ clumps_get_raw_info(struct clumps_thread_params *cltprm)
{
coord[0]=GAL_DIMENSION_FLT_TO_INT(row[INFO_X]/row[INFO_SFF]);
coord[1]=GAL_DIMENSION_FLT_TO_INT(row[INFO_Y]/row[INFO_SFF]);
- row[INFO_INSTD] = std[ gal_tile_full_id_from_coord(&p->cp.tl,
- coord) ];
+ row[INFO_INSTD]=( p->std->size==p->input->size
+ ? std[ gal_dimension_coord_to_index(ndim,
+ dsize,
+ coord) ]
+ : std[ gal_tile_full_id_from_coord(tl,
+ coord)] );
/* For a check
printf("---------\n");
printf("\t%f --> %zu\n", row[INFO_Y]/row[INFO_SFF], coord[1]);
@@ -822,7 +350,7 @@ clumps_get_raw_info(struct clumps_thread_params *cltprm)
void
clumps_make_sn_table(struct clumps_thread_params *cltprm)
{
- struct noisechiselparams *p=cltprm->clprm->p;
+ struct segmentparams *p=cltprm->clprm->p;
size_t tablen=cltprm->numinitclumps+1;
float *snarr;
@@ -888,13 +416,10 @@ clumps_make_sn_table(struct clumps_thread_params *cltprm)
noise cases) or the area is smaller than the minimum area to
calculate signal-to-noise, then set the S/N of this segment to
zero. */
- if( I>O && Ni>p->segsnminarea )
+ if( I>O && Ni>p->snminarea )
{
- /* Here we have done sky subtraction once. However, if the sky
- was already subtracted (informed by the user), then the
- varience should be multiplied by 2. */
- var = ( (p->skysubtracted ? 2.0f : 1.0f)
- * row[INFO_INSTD] * row[INFO_INSTD] );
+ /* For easy reading, define `var' for variance. */
+ var = row[INFO_INSTD] * row[INFO_INSTD];
/* Calculate the Signal to noise ratio, if we are on the noise
regions, we don't care about the IDs of the clumps anymore, so
@@ -943,7 +468,7 @@ clumps_correct_sky_labels_for_check(struct
clumps_thread_params *cltprm,
gal_data_t *newinds;
int32_t *ninds, curlab, *l, *lf;
size_t len=cltprm->numinitclumps+1;
- struct noisechiselparams *p=cltprm->clprm->p;
+ struct segmentparams *p=cltprm->clprm->p;
/* If there are no clumps in this tile, then this function can be
ignored. */
@@ -975,9 +500,10 @@ clumps_correct_sky_labels_for_check(struct
clumps_thread_params *cltprm,
if(p->cp.numthreads>1) pthread_mutex_unlock(&cltprm->clprm->labmutex);
- /* Initialize the newinds array to CLUMPS_INIT (which be used as a new
+ /* Initialize the newinds array to GAL_LABEL_INIT (which be used as a new
label for all the clumps that must be removed. */
- lf = (l=newinds->array) + newinds->size; do *l++=CLUMPS_INIT; while(l<lf);
+ lf = (l=newinds->array) + newinds->size;
+ do *l++=GAL_LABEL_INIT; while(l<lf);
/* The new indexs array has been initialized to zero. So we just need to
@@ -1005,7 +531,7 @@ clumps_find_make_sn_table(void *in_prm)
{
struct gal_threads_params *tprm=(struct gal_threads_params *)in_prm;
struct clumps_params *clprm=(struct clumps_params *)(tprm->params);
- struct noisechiselparams *p=clprm->p;
+ struct segmentparams *p=clprm->p;
size_t ndim=p->input->ndim, *dsize=p->input->dsize;
void *tarray;
@@ -1043,7 +569,7 @@ clumps_find_make_sn_table(void *in_prm)
/* Get the number of usable elements in this tile (note that tiles
can have blank pixels), so we can't simply use `tile->size'. */
- if(tile->flag & GAL_DATA_FLAG_HASBLANK)
+ if(p->input->flag & GAL_DATA_FLAG_HASBLANK)
{
tmp=gal_statistics_number(tile);
num=*((size_t *)(tmp->array));
@@ -1114,7 +640,7 @@ clumps_find_make_sn_table(void *in_prm)
|| icoord[0]==scoord[0]+tile->dsize[0]-1
|| icoord[1]==scoord[1]
|| icoord[1]==scoord[1]+tile->dsize[1]-1 )
- *(int32_t *)i=CLUMPS_RIVER;
+ *(int32_t *)i=GAL_LABEL_RIVER;
/* This pixel is not on the edge, check if it had a value of
`0' in the binary image (is not detected) then add it to
@@ -1151,13 +677,15 @@ clumps_find_make_sn_table(void *in_prm)
/* Generate the clumps over this region. */
- clumps_oversegment(&cltprm);
+ cltprm.numinitclumps=gal_label_oversegment(p->conv, cltprm.indexs,
+ p->clabel,
+ cltprm.topinds);
- /* Set all river pixels to CLUMPS_INIT (to be distinguishable
+ /* Set all river pixels to GAL_LABEL_INIT (to be distinguishable
from the detected regions). */
GAL_TILE_PO_OISET( int32_t, int, tile, NULL, 0, 1,
- {if(*i==CLUMPS_RIVER) *i=CLUMPS_INIT;} );
+ {if(*i==GAL_LABEL_RIVER) *i=GAL_LABEL_INIT;} );
/* For a check, the step variable will be set. */
@@ -1198,6 +726,59 @@ clumps_find_make_sn_table(void *in_prm)
+/* Write the S/N table. */
+static void
+clumps_write_sn_table(struct segmentparams *p, gal_data_t *insn,
+ gal_data_t *inind, char *filename,
+ gal_list_str_t *comments)
+{
+ gal_data_t *sn, *ind, *cols;
+
+ /* Remove all blank elements. The index and sn values must have the same
+ set of blank elements, but checking on the integer array is faster. */
+ if( gal_blank_present(inind, 1) )
+ {
+ /* Remove blank elements. */
+ ind=gal_data_copy(inind);
+ sn=gal_data_copy(insn);
+ gal_blank_remove(ind);
+ gal_blank_remove(sn);
+
+ /* A small sanity check. */
+ if(ind->size==0 || sn->size==0)
+ error(EXIT_FAILURE, 0, "%s: a bug! Please contact us at %s to fix "
+ "the problem. For some reason, all the elements in `ind' or "
+ "`sn' are blank", __func__, PACKAGE_BUGREPORT);
+ }
+ else
+ {
+ sn = insn;
+ ind = inind;
+ }
+
+ /* Set the columns. */
+ cols = ind;
+ cols->next = sn;
+
+
+ /* Prepare the comments. */
+ gal_table_comments_add_intro(&comments, PROGRAM_STRING, &p->rawtime);
+
+
+ /* write the table. */
+ gal_checkset_writable_remove(filename, 0, 1);
+ gal_table_write(cols, comments, p->cp.tableformat, filename, "SN");
+
+
+ /* Clean up (if necessary). */
+ if(sn!=insn) gal_data_free(sn);
+ if(ind==inind) ind->next=NULL; else gal_data_free(ind);
+}
+
+
+
+
+
/* Find the true clump signal to noise value from the clumps in the sky
region.
@@ -1214,7 +795,7 @@ clumps_find_make_sn_table(void *in_prm)
concatenate all the S/N values into one array and send it to the main
findsnthresh function in thresh.c. */
void
-clumps_true_find_sn_thresh(struct noisechiselparams *p)
+clumps_true_find_sn_thresh(struct segmentparams *p)
{
char *msg;
struct timeval t1;
@@ -1233,7 +814,7 @@ clumps_true_find_sn_thresh(struct noisechiselparams *p)
clprm.p=p;
clprm.sky0_det1=0;
clprm.sn=gal_data_array_calloc(p->ltl.tottiles);
- clprm.snind = ( p->checksegmentation || p->checkclumpsn
+ clprm.snind = ( p->checksegmentation || p->checksn
? gal_data_array_calloc(p->ltl.tottiles) : NULL );
@@ -1243,7 +824,7 @@ clumps_true_find_sn_thresh(struct noisechiselparams *p)
label. Since `p->numclumps' is not used yet, we will use it here. When
multiple threads are used, we will need a mutex to make sure that only
one thread can change this central variable at every one moment. */
- if(p->checksegmentation || p->checkclumpsn)
+ if(p->checksegmentation || p->checksn)
{
p->numclumps=0;
if( p->cp.numthreads > 1 ) pthread_mutex_init(&clprm.labmutex, NULL);
@@ -1310,7 +891,7 @@ clumps_true_find_sn_thresh(struct noisechiselparams *p)
/* Destroy the mutex if it was initialized. */
- if( p->cp.numthreads>1 && (p->checksegmentation || p->checkclumpsn) )
+ if( p->cp.numthreads>1 && (p->checksegmentation || p->checksn) )
pthread_mutex_destroy(&clprm.labmutex);
@@ -1330,7 +911,7 @@ clumps_true_find_sn_thresh(struct noisechiselparams *p)
sn=gal_data_alloc(NULL, GAL_TYPE_FLOAT32, 1, &numsn, NULL, 0,
p->cp.minmapsize, "CLUMP_S/N", "ratio",
"Signal-to-noise ratio");
- snind = ( p->checkclumpsn
+ snind = ( p->checksn
? gal_data_alloc(NULL, GAL_TYPE_INT32, 1, &numsn, NULL, 0,
p->cp.minmapsize, "CLUMP_ID", "counter",
"Unique ID for this clump.")
@@ -1357,7 +938,7 @@ clumps_true_find_sn_thresh(struct noisechiselparams *p)
/* If the user wanted to see the S/N table, then save it. */
- if(p->checkclumpsn)
+ if(p->checksn)
{
/* Make the comments, then write the table and free the comments. */
if(p->cp.numthreads>1)
@@ -1367,18 +948,18 @@ clumps_true_find_sn_thresh(struct noisechiselparams *p)
"output with `--checksegmentation'.", 1);
gal_list_str_add(&comments, "S/N of clumps over undetected regions.",
1);
- threshold_write_sn_table(p, sn, snind, p->clumpsn_s_name, comments);
+ clumps_write_sn_table(p, sn, snind, p->clumpsn_s_name, comments);
gal_list_str_free(comments, 1);
}
/* Find the desired quantile from the full S/N distribution. */
- quant = gal_statistics_quantile(sn, p->segquant, 1);
+ quant = gal_statistics_quantile(sn, p->snquant, 1);
p->clumpsnthresh = *((float *)(quant->array));
if(!p->cp.quiet)
{
if( asprintf(&msg, "Clump S/N: %.2f (%.3f quant of %zu).",
- p->clumpsnthresh, p->segquant, sn->size)<0 )
+ p->clumpsnthresh, p->snquant, sn->size)<0 )
error(EXIT_FAILURE, 0, "%s: asprintf allocation", __func__);
gal_timing_report(&t1, msg, 2);
free(msg);
@@ -1417,7 +998,7 @@ clumps_true_find_sn_thresh(struct noisechiselparams *p)
(where each element is a dataset containing the respective label's
indexs). */
gal_data_t *
-clumps_det_label_indexs(struct noisechiselparams *p)
+clumps_det_label_indexs(struct segmentparams *p)
{
size_t i, *areas;
int32_t *a, *l, *lf;
@@ -1471,7 +1052,7 @@ clumps_det_label_indexs(struct noisechiselparams *p)
void
clumps_det_keep_true_relabel(struct clumps_thread_params *cltprm)
{
- struct noisechiselparams *p=cltprm->clprm->p;
+ struct segmentparams *p=cltprm->clprm->p;
size_t ndim=p->input->ndim, *dsize=p->input->dsize;
int istouching;
@@ -1490,10 +1071,10 @@ clumps_det_keep_true_relabel(struct
clumps_thread_params *cltprm)
"newlabs");
dinc=gal_dimension_increment(ndim, dsize);
- /* Initialize the new labels with CLUMPS_INIT (so the diffuse area
+ /* Initialize the new labels with GAL_LABEL_INIT (so the diffuse area
can be distinguished from the clumps). */
lf=(l=newlabs)+cltprm->numinitclumps+1;
- do *l++=CLUMPS_INIT; while(l<lf);
+ do *l++=GAL_LABEL_INIT; while(l<lf);
/* Set the new labels. Here we will also be removing clumps with a peak
that touches a river pixel. */
@@ -1523,10 +1104,10 @@ clumps_det_keep_true_relabel(struct
clumps_thread_params *cltprm)
}
}
- /* Correct the clump labels. Note that the non-clumpy regions over the
- detections (rivers) have already been initialized to CLUMPS_INIT
- (which is negative). So we'll just need to correct the ones with a
- value larger than 0. */
+ /* Correct the clump labels. Note that the non-clumpy regions over
+ the detections (rivers) have already been initialized to
+ GAL_LABEL_INIT (which is negative). So we'll just need to correct
+ the ones with a value larger than 0. */
sf=(s=cltprm->indexs->array)+cltprm->indexs->size;
do if(clabel[*s]>0) clabel[*s] = newlabs[ clabel[*s] ]; while(++s<sf);
diff --git a/bin/noisechisel/clumps.h b/bin/segment/clumps.h
similarity index 86%
rename from bin/noisechisel/clumps.h
rename to bin/segment/clumps.h
index 531f6c3..9799c45 100644
--- a/bin/noisechisel/clumps.h
+++ b/bin/segment/clumps.h
@@ -1,6 +1,6 @@
/*********************************************************************
-NoiseChisel - Detect and segment signal in a noisy dataset.
-NoiseChisel is part of GNU Astronomy Utilities (Gnuastro) package.
+Segment - Segment initial labels based on signal structure.
+Segment is part of GNU Astronomy Utilities (Gnuastro) package.
Original author:
Mohammad Akhlaghi <address@hidden>
@@ -24,11 +24,6 @@ along with Gnuastro. If not, see
<http://www.gnu.org/licenses/>.
#define CLUMPS_H
-/* Constants for the clump over-segmentation. */
-#define CLUMPS_INIT -1
-#define CLUMPS_RIVER -2
-#define CLUMPS_TMPCHECK -3
-
/* Parameters for all threads. */
struct clumps_params
@@ -36,7 +31,7 @@ struct clumps_params
/* General */
int step; /* Counter if we want to check steps. */
int sky0_det1; /* If working on the Sky or Detections. */
- struct noisechiselparams *p; /* Pointer to main NoiseChisel parameters. */
+ struct segmentparams *p; /* Pointer to main Segment parameters. */
pthread_mutex_t labmutex; /* Mutex to change the total numbers. */
/* For Sky region. */
@@ -68,10 +63,6 @@ struct clumps_thread_params
struct clumps_params *clprm; /* Pointer to main structure. */
};
-
-void
-clumps_oversegment(struct clumps_thread_params *cltprm);
-
void
clumps_grow_prepare_initial(struct clumps_thread_params *cltprm);
@@ -83,13 +74,13 @@ clumps_grow(gal_data_t *labels, gal_data_t *diffuseindexs,
int withrivers,
int connectivity);
void
-clumps_true_find_sn_thresh(struct noisechiselparams *p);
+clumps_true_find_sn_thresh(struct segmentparams *p);
void
clumps_make_sn_table(struct clumps_thread_params *cltprm);
gal_data_t *
-clumps_det_label_indexs(struct noisechiselparams *p);
+clumps_det_label_indexs(struct segmentparams *p);
void
clumps_det_keep_true_relabel(struct clumps_thread_params *cltprm);
diff --git a/bin/noisechisel/main.c b/bin/segment/main.c
similarity index 79%
copy from bin/noisechisel/main.c
copy to bin/segment/main.c
index c49c91b..34254a3 100644
--- a/bin/noisechisel/main.c
+++ b/bin/segment/main.c
@@ -1,11 +1,11 @@
/*********************************************************************
-NoiseChisel - Detect and segment signal in a noisy dataset.
-NoiseChisel is part of GNU Astronomy Utilities (Gnuastro) package.
+Segment - Segment initial labels based on signal structure.
+Segment is part of GNU Astronomy Utilities (Gnuastro) package.
Original author:
Mohammad Akhlaghi <address@hidden>
Contributing author(s):
-Copyright (C) 2015-2018, Free Software Foundation, Inc.
+Copyright (C) 2018, Free Software Foundation, Inc.
Gnuastro is free software: you can redistribute it and/or modify it
under the terms of the GNU General Public License as published by the
@@ -30,7 +30,7 @@ along with Gnuastro. If not, see
<http://www.gnu.org/licenses/>.
#include "main.h"
#include "ui.h"
-#include "noisechisel.h"
+#include "segment.h"
/* Main function */
@@ -38,17 +38,17 @@ int
main (int argc, char *argv[])
{
struct timeval t1;
- struct noisechiselparams p={{{0},0},{0},0};
+ struct segmentparams p={{{0},0},{0},0};
- /* Set they starting time. */
+ /* Set the starting time. */
time(&p.rawtime);
gettimeofday(&t1, NULL);
/* Read the input parameters. */
ui_read_check_inputs_setup(argc, argv, &p);
- /* Run MakeProfiles */
- noisechisel(&p);
+ /* Run Segment */
+ segment(&p);
/* Free all non-freed allocations. */
ui_free_report(&p, &t1);
diff --git a/bin/segment/main.h b/bin/segment/main.h
new file mode 100644
index 0000000..76e2335
--- /dev/null
+++ b/bin/segment/main.h
@@ -0,0 +1,104 @@
+/*********************************************************************
+Segment - Segment initial labels based on signal structure.
+Segment is part of GNU Astronomy Utilities (Gnuastro) package.
+
+Original author:
+ Mohammad Akhlaghi <address@hidden>
+Contributing author(s):
+Copyright (C) 2018, Free Software Foundation, Inc.
+
+Gnuastro is free software: you can redistribute it and/or modify it
+under the terms of the GNU General Public License as published by the
+Free Software Foundation, either version 3 of the License, or (at your
+option) any later version.
+
+Gnuastro is distributed in the hope that it will be useful, but
+WITHOUT ANY WARRANTY; without even the implied warranty of
+MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the GNU
+General Public License for more details.
+
+You should have received a copy of the GNU General Public License
+along with Gnuastro. If not, see <http://www.gnu.org/licenses/>.
+**********************************************************************/
+#ifndef MAIN_H
+#define MAIN_H
+
+/* Include necessary headers */
+#include <gnuastro/data.h>
+
+#include <gnuastro-internal/options.h>
+
+/* Progarm names. */
+#define PROGRAM_NAME "Segment" /* Program full name. */
+#define PROGRAM_EXEC "astsegment" /* Program executable name. */
+#define PROGRAM_STRING PROGRAM_NAME" (" PACKAGE_NAME ") " PACKAGE_VERSION
+
+
+
+
+
+
+
+/* Main program parameters structure */
+struct segmentparams
+{
+ /* From command-line */
+ struct gal_options_common_params cp; /* Common parameters. */
+ struct gal_tile_two_layer_params ltl; /* Large tessellation. */
+ char *inputname; /* Input filename. */
+ char *kernelname; /* Input kernel filename. */
+ char *khdu; /* Kernel HDU. */
+ char *convolvedname; /* Convolved image (to avoid convolution).*/
+ char *chdu; /* HDU of convolved image. */
+ char *detectionname; /* Detection image file name. */
+ char *dhdu; /* Detection image file name. */
+ char *skyname; /* Filename of Sky image. */
+ char *skyhdu; /* Filename of Sky image. */
+ char *stdname; /* File name of Standard deviation image. */
+ char *stdhdu; /* HDU of Stanard deviation image. */
+ uint8_t rawoutput; /* Output only object and clump labels. */
+
+ float minskyfrac; /* Undetected area min. frac. in tile. */
+ size_t snminarea; /* Minimum area for segmentation. */
+ uint8_t checksn; /* Save the clump S/N values to a file. */
+ size_t minnumfalse; /* Min No. of det/seg for true quantile. */
+ float snquant; /* Quantile of clumps in sky for true S/N.*/
+ uint8_t keepmaxnearriver; /* Keep clumps with a peak near a river. */
+ float gthresh; /* Multiple of STD to stop growing clumps.*/
+ size_t minriverlength; /* Min, len of good grown clump rivers. */
+ float objbordersn; /* Minimum S/N for grown clumps to be one.*/
+ uint8_t grownclumps; /* Save grown clumps instead of original. */
+ uint8_t continueaftercheck; /* Don't abort after the check steps. */
+ uint8_t checksegmentation; /* Save the segmentation steps in file. */
+
+ /* Internal. */
+ char *clumpsn_s_name; /* Sky clump S/N name. */
+ char *clumpsn_d_name; /* Detection clumps S/N name. */
+ char *segmentationname; /* Name of segmentation steps file. */
+
+ gal_data_t *input; /* Input dataset. */
+ gal_data_t *kernel; /* Given kernel for convolution. */
+ gal_data_t *conv; /* Convolved dataset. */
+ gal_data_t *binary; /* For binary operations. */
+ gal_data_t *olabel; /* Object labels. */
+ gal_data_t *clabel; /* Clumps labels. */
+ gal_data_t *std; /* STD of undetected pixels, per tile. */
+
+ float cpscorr; /* Counts/second correction. */
+ size_t numdetections; /* Number of final detections. */
+ size_t numclumps; /* Number of true clumps. */
+ size_t numobjects; /* Number of objects. */
+ float clumpsnthresh; /* Clump S/N threshold. */
+
+ char *useddetectionname; /* Name of file USED for detection image. */
+ char *usedstdname; /* Name of file USED for sky STD image. */
+
+ float medstd; /* For output STD image: median STD. */
+ float minstd; /* For output STD image: median STD. */
+ float maxstd; /* For output STD image: median STD. */
+
+ /* Output: */
+ time_t rawtime; /* Starting time of the program. */
+};
+
+#endif
diff --git a/bin/noisechisel/segmentation.c b/bin/segment/segment.c
similarity index 76%
rename from bin/noisechisel/segmentation.c
rename to bin/segment/segment.c
index c54453b..caaca61 100644
--- a/bin/noisechisel/segmentation.c
+++ b/bin/segment/segment.c
@@ -1,6 +1,6 @@
/*********************************************************************
-NoiseChisel - Detect and segment signal in a noisy dataset.
-NoiseChisel is part of GNU Astronomy Utilities (Gnuastro) package.
+Segment - Segment initial labels based on signal structure.
+Segment is part of GNU Astronomy Utilities (Gnuastro) package.
Original author:
Mohammad Akhlaghi <address@hidden>
@@ -28,11 +28,15 @@ along with Gnuastro. If not, see
<http://www.gnu.org/licenses/>.
#include <stdlib.h>
#include <string.h>
+#include <gnuastro/wcs.h>
#include <gnuastro/fits.h>
#include <gnuastro/blank.h>
+#include <gnuastro/label.h>
#include <gnuastro/binary.h>
#include <gnuastro/threads.h>
+#include <gnuastro/convolve.h>
#include <gnuastro/dimension.h>
+#include <gnuastro/statistics.h>
#include <gnuastro-internal/timing.h>
#include <gnuastro-internal/checkset.h>
@@ -41,7 +45,155 @@ along with Gnuastro. If not, see
<http://www.gnu.org/licenses/>.
#include "ui.h"
#include "clumps.h"
-#include "segmentation.h"
+#include "segment.h"
+
+
+
+
+
+
+
+
+
+
+/***********************************************************************/
+/***************** Preparations *****************/
+/***********************************************************************/
+static void
+segment_convolve(struct segmentparams *p)
+{
+ struct timeval t1;
+ struct gal_tile_two_layer_params *tl=&p->cp.tl;
+
+ /* Convovle with sharper kernel. */
+ if(p->conv==NULL)
+ {
+ /* Do the convolution if a kernel was requested. */
+ if(p->kernel)
+ {
+ /* Make the convolved image. */
+ if(!p->cp.quiet) gettimeofday(&t1, NULL);
+ p->conv = gal_convolve_spatial(tl->tiles, p->kernel,
+ p->cp.numthreads, 1, tl->workoverch);
+
+ /* Report and write check images if necessary. */
+ if(!p->cp.quiet)
+ gal_timing_report(&t1, "Convolved with given kernel.", 1);
+ }
+ else
+ p->conv=p->input;
+ }
+
+ /* Make necessary corrections to the convolved array. */
+ if(p->conv!=p->input)
+ {
+ /* Set the flags (most importantly, the blank flags). */
+ p->conv->flag = p->input->flag;
+
+ /* Set the name. */
+ if(p->conv->name) free(p->conv->name);
+ gal_checkset_allocate_copy("CONVOLVED", &p->conv->name);
+ }
+}
+
+
+
+
+
+static void
+segment_initialize(struct segmentparams *p)
+{
+ uint8_t *b;
+ float *f, minv;
+ gal_data_t *min;
+ gal_data_t *forcc;
+ int32_t *o, *c, *cf, maxlab=0;
+
+ /* Allocate the clump labels image and the binary image. */
+ p->clabel=gal_data_alloc(NULL, p->olabel->type, p->olabel->ndim,
+ p->olabel->dsize, p->olabel->wcs, 1,
+ p->cp.minmapsize, NULL, NULL, NULL);
+ p->binary=gal_data_alloc(NULL, GAL_TYPE_UINT8, p->olabel->ndim,
+ p->olabel->dsize, p->olabel->wcs, 1,
+ p->cp.minmapsize, NULL, NULL, NULL);
+ p->clabel->flag=p->input->flag;
+ p->binary->wcs=gal_wcs_copy(p->input->wcs);
+ p->clabel->wcs=gal_wcs_copy(p->input->wcs);
+
+
+ /* Prepare the `binary', `clabel' and `olabel' arrays. */
+ b=p->binary->array;
+ o=p->olabel->array;
+ f=p->input->array; cf=(c=p->clabel->array)+p->clabel->size;
+ do
+ {
+ if(isnan(*f++)) *o = *c = GAL_BLANK_INT32;
+ else
+ {
+ /* Initialize the binary array. */
+ *b = *o > 0;
+
+ /* A small sanity check. */
+ if(*o<0)
+ error(EXIT_FAILURE, 0, "%s (hdu: %s) has negative value(s). "
+ "Each non-zero pixel in this image must be positive (a "
+ "counter, counting from 1).", p->useddetectionname,
+ p->dhdu);
+
+ /* Set the largest value. */
+ if(*o>maxlab) maxlab=*o;
+ }
+ ++o;
+ ++b;
+ }
+ while(++c<cf);
+
+ /* prepare the labeled image. */
+ switch(maxlab)
+ {
+ /* No positive values. */
+ case 0:
+ error(EXIT_FAILURE, 0, "%s (hdu: %s): contains no detections "
+ "(positive valued pixels)", p->useddetectionname, p->dhdu);
+
+ /* This is a binary image, probably it was the result of a
+ threshold. In this case, we need to separate the connected
+ components. */
+ case 1:
+ forcc=gal_data_copy_to_new_type(p->olabel, GAL_TYPE_UINT8);
+ p->numdetections=gal_binary_connected_components(forcc, &p->olabel,
+ p->olabel->ndim);
+ break;
+
+ /* There is more than one label in the image, so just set the maximum
+ number to the number of detections. */
+ default:
+ p->numdetections=maxlab;
+ }
+
+ /* If the minimum standard deviation is less than 1, then the units of
+ the input are in units of counts/time. As described in the NoiseChisel
+ paper, we need to correct the S/N equation later. */
+ min=gal_statistics_minimum(p->std);
+ minv=*(float *)(min->array);
+ p->cpscorr = minv>1 ? 1.0 : minv;
+ gal_data_free(min);
+}
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
@@ -55,7 +207,7 @@ along with Gnuastro. If not, see
<http://www.gnu.org/licenses/>.
composed of multiple objects). So the labels within each detection start
from 1.*/
static void
-segmentation_relab_noseg(struct clumps_thread_params *cltprm)
+segment_relab_noseg(struct clumps_thread_params *cltprm)
{
int32_t *olabel=cltprm->clprm->p->olabel->array;
size_t *s=cltprm->indexs->array, *sf=s+cltprm->indexs->size;
@@ -76,10 +228,10 @@ segmentation_relab_noseg(struct clumps_thread_params
*cltprm)
a*numobjs+b or b*numobjs+a and get the answer. Since the number of
objects in a given region will not be too high, this is efficient. */
static void
-segmentation_relab_to_objects(struct clumps_thread_params *cltprm)
+segment_relab_to_objects(struct clumps_thread_params *cltprm)
{
size_t amwidth=cltprm->numtrueclumps+1;
- struct noisechiselparams *p=cltprm->clprm->p;
+ struct segmentparams *p=cltprm->clprm->p;
size_t ndim=p->input->ndim, *dsize=p->input->dsize;
size_t mdsize[2]={amwidth, amwidth};
@@ -91,13 +243,13 @@ segmentation_relab_to_objects(struct clumps_thread_params
*cltprm)
NULL, 1, p->cp.minmapsize, NULL,
NULL, NULL);
float *imgss=p->input->array;
+ double var=cltprm->std*cltprm->std;
uint8_t *adjacency=adjacency_d->array;
size_t nngb=gal_dimension_num_neighbors(ndim);
int32_t *clumptoobj, *olabel=p->olabel->array;
size_t *dinc=gal_dimension_increment(ndim, dsize);
size_t *s, *sf, i, j, ii, rpnum, *nums=nums_d->array;
double ave, rpsum, c=sqrt(1/p->cpscorr), *sums=sums_d->array;
- double err=cltprm->std*cltprm->std*(p->skysubtracted?1.0f:2.0f);
int32_t *ngblabs=gal_data_malloc_array(GAL_TYPE_UINT32, nngb, __func__,
"ngblabs");
@@ -114,7 +266,7 @@ segmentation_relab_to_objects(struct clumps_thread_params
*cltprm)
do
/* We only want to work on pixels that have already been identified
as touching more than one label: river pixels. */
- if( olabel[ *s ]==CLUMPS_RIVER )
+ if( olabel[ *s ]==GAL_LABEL_RIVER )
{
/* Initialize the values. */
i=ii=0;
@@ -188,7 +340,7 @@ segmentation_relab_to_objects(struct clumps_thread_params
*cltprm)
acceptable, and we put no area criteria here, because
the fact that a river exists between two clumps is
important. */
- if( ave>0.0f && ( c * ave / sqrt(ave+err) ) > p->objbordersn )
+ if( ave>0.0f && ( c * ave / sqrt(ave+var) ) > p->objbordersn )
{
adjacency[ii]=1; /* We want to set both sides of the */
adjacency[ j * amwidth + i ] = 1; /* Symmetric matrix. */
@@ -213,7 +365,7 @@ segmentation_relab_to_objects(struct clumps_thread_params
*cltprm)
ave=sums[ii]/nums[ii];
printf(" ...%zu: N:%-4zu S:%-10.2f S/N: %-10.2f "
"--> %u\n", j, nums[ii], sums[ii],
- c*ave/sqrt(ave+err), adjacency[ii]);
+ c*ave/sqrt(ave+var), adjacency[ii]);
}
}
printf("\n");
@@ -283,7 +435,7 @@ segmentation_relab_to_objects(struct clumps_thread_params
*cltprm)
this function, we want to correct the clump labels so the clump IDs in
each object start from 1 and are contiguous. */
static void
-segmentation_relab_clumps_in_objects(struct clumps_thread_params *cltprm)
+segment_relab_clumps_in_objects(struct clumps_thread_params *cltprm)
{
size_t numobjects=cltprm->numobjects, numtrueclumps=cltprm->numtrueclumps;
@@ -319,7 +471,7 @@ segmentation_relab_clumps_in_objects(struct
clumps_thread_params *cltprm)
instance, this function will use a mutex to limit the reading and
writing to the variable keeping the total number of objects counter. */
static void
-segmentation_relab_overall(struct clumps_thread_params *cltprm)
+segment_relab_overall(struct clumps_thread_params *cltprm)
{
struct clumps_params *clprm=cltprm->clprm;
int32_t startinglab, *olabel=clprm->p->olabel->array;
@@ -369,11 +521,11 @@ segmentation_relab_overall(struct clumps_thread_params
*cltprm)
/***********************************************************************/
/* Find the true clumps over each detection. */
static void *
-segmentation_on_threads(void *in_prm)
+segment_on_threads(void *in_prm)
{
struct gal_threads_params *tprm=(struct gal_threads_params *)in_prm;
struct clumps_params *clprm=(struct clumps_params *)(tprm->params);
- struct noisechiselparams *p=clprm->p;
+ struct segmentparams *p=clprm->p;
size_t i, *s, *sf;
gal_data_t *topinds;
@@ -410,7 +562,16 @@ segmentation_on_threads(void *in_prm)
/* Find the clumps over this region. */
- clumps_oversegment(&cltprm);
+ cltprm.numinitclumps=gal_label_oversegment(p->conv, cltprm.indexs,
+ p->clabel, cltprm.topinds);
+
+
+ /* Set all the river pixels to zero (we don't need them any more in
+ the clumps image). */
+ sf=(s=cltprm.indexs->array) + cltprm.indexs->size;
+ do
+ if( clabel[*s]==GAL_LABEL_RIVER ) clabel[*s]=GAL_LABEL_INIT;
+ while(++s<sf);
/* Make the clump S/N table. This table is made before (possibly)
@@ -430,7 +591,7 @@ segmentation_on_threads(void *in_prm)
/* If the user wanted to check the segmentation steps or the clump
S/N values in a table, then we have to stop the process at this
point. */
- if(clprm->step==1 || p->checkclumpsn)
+ if(clprm->step==1 || p->checksn)
{ gal_data_free(topinds); continue; }
/* Only keep true clumps. */
@@ -450,7 +611,7 @@ segmentation_on_threads(void *in_prm)
{
/* Set the basics. */
cltprm.numobjects=1;
- segmentation_relab_noseg(&cltprm);
+ segment_relab_noseg(&cltprm);
/* If the user wanted a check image, this object doesn't
change. */
@@ -472,7 +633,7 @@ segmentation_on_threads(void *in_prm)
/* Grow the true clumps over the detection. */
clumps_grow_prepare_initial(&cltprm);
if(cltprm.diffuseindexs->size)
- clumps_grow(p->olabel, cltprm.diffuseindexs, 1, 1);
+ gal_label_grow_indexs(p->olabel, cltprm.diffuseindexs, 1, 1);
if(clprm->step==3)
{ gal_data_free(cltprm.diffuseindexs); continue; }
@@ -488,7 +649,7 @@ segmentation_on_threads(void *in_prm)
/* Identify the objects in this detection using the grown clumps
and correct the grown clump labels into new object labels. */
- segmentation_relab_to_objects(&cltprm);
+ segment_relab_to_objects(&cltprm);
if(clprm->step==4)
{
gal_data_free(cltprm.clumptoobj);
@@ -500,7 +661,7 @@ segmentation_on_threads(void *in_prm)
the diffuse indexs any more, so after filling the detected
region, free the indexs. */
if( cltprm.numobjects == 1 )
- segmentation_relab_noseg(&cltprm);
+ segment_relab_noseg(&cltprm);
else
{
/* Correct the labels so every non-labeled pixel can be
@@ -509,8 +670,8 @@ segmentation_on_threads(void *in_prm)
/* Cover the whole area (using maximum connectivity to not
miss any pixels). */
- clumps_grow(p->olabel, cltprm.diffuseindexs, 0,
- p->olabel->ndim);
+ gal_label_grow_indexs(p->olabel, cltprm.diffuseindexs, 0,
+ p->olabel->ndim);
/* Make sure all diffuse pixels are labeled. */
if(cltprm.diffuseindexs->size)
@@ -526,13 +687,13 @@ segmentation_on_threads(void *in_prm)
when there is more than object over the detection or when
there were multiple clumps over the detection. */
if(cltprm.numobjects>1)
- segmentation_relab_clumps_in_objects(&cltprm);
+ segment_relab_clumps_in_objects(&cltprm);
gal_data_free(cltprm.clumptoobj);
if(clprm->step==6) {continue;}
}
/* Convert the object labels to their final value */
- segmentation_relab_overall(&cltprm);
+ segment_relab_overall(&cltprm);
}
/* Wait until all the threads finish then return. */
@@ -547,7 +708,7 @@ segmentation_on_threads(void *in_prm)
/* If the user wanted to see the S/N table in a file, this function will be
called and will do the job. */
static void
-segmentation_save_sn_table(struct clumps_params *clprm)
+segment_save_sn_table(struct clumps_params *clprm)
{
char *msg;
float *sarr;
@@ -555,7 +716,7 @@ segmentation_save_sn_table(struct clumps_params *clprm)
gal_list_str_t *comments=NULL;
size_t i, j, c=0, totclumps=0;
gal_data_t *sn, *objind, *clumpinobj;
- struct noisechiselparams *p=clprm->p;
+ struct segmentparams *p=clprm->p;
/* Find the total number of clumps in all the initial detections. Recall
@@ -594,7 +755,7 @@ segmentation_save_sn_table(struct clumps_params *clprm)
gal_list_str_add(&comments, "See also: `CLUMPS_ALL_DET' HDU of "
"output with `--checksegmentation'.", 1);
if( asprintf(&msg, "S/N values of `nan': clumps smaller than "
- "`--segsnminarea' of %zu.", p->segsnminarea)<0 )
+ "`--snminarea' of %zu.", p->snminarea)<0 )
error(EXIT_FAILURE, 0, "%s: asprintf allocation", __func__);
gal_list_str_add(&comments, msg, 0);
gal_list_str_add(&comments, "S/N of clumps over detected regions.", 1);
@@ -628,7 +789,7 @@ segmentation_save_sn_table(struct clumps_params *clprm)
/* Find true clumps over the detected regions. */
static void
-segmentation_detections(struct noisechiselparams *p)
+segment_detections(struct segmentparams *p)
{
char *msg;
struct clumps_params clprm;
@@ -677,7 +838,7 @@ segmentation_detections(struct noisechiselparams *p)
claborig->size*gal_type_sizeof(claborig->type));
/* (Re-)do everything until this step. */
- gal_threads_spin_off(segmentation_on_threads, &clprm,
+ gal_threads_spin_off(segment_on_threads, &clprm,
p->numdetections, p->cp.numthreads);
/* Set the extension name. */
@@ -716,7 +877,8 @@ segmentation_detections(struct noisechiselparams *p)
demo->name = "DET_CLUMPS_GROWN";
if(!p->cp.quiet)
{
- gal_timing_report(NULL, "Starting to identify objects.", 1);
+ gal_timing_report(NULL, "Identify objects...",
+ 1);
if( asprintf(&msg, "True clumps grown "
"(HDU: `%s').", demo->name)<0 )
error(EXIT_FAILURE, 0, "%s: asprintf allocation",
@@ -799,7 +961,7 @@ segmentation_detections(struct noisechiselparams *p)
out of the loop, we don't need the rest of the process any
more. */
if( clprm.step==1
- && ( p->checkclumpsn && !p->continueaftercheck ) ) break;
+ && ( p->checksn && !p->continueaftercheck ) ) break;
/* Increment the step counter. */
++clprm.step;
@@ -812,7 +974,7 @@ segmentation_detections(struct noisechiselparams *p)
else
{
clprm.step=0;
- gal_threads_spin_off(segmentation_on_threads, &clprm, p->numdetections,
+ gal_threads_spin_off(segment_on_threads, &clprm, p->numdetections,
p->cp.numthreads);
}
@@ -822,7 +984,7 @@ segmentation_detections(struct noisechiselparams *p)
p->numobjects=clprm.totobjects;
/* If the user wanted to see the S/N table, then make the S/N table. */
- if(p->checkclumpsn) segmentation_save_sn_table(&clprm);
+ if(p->checksn) segment_save_sn_table(&clprm);
/* Clean up allocated structures and destroy the mutex. */
@@ -851,27 +1013,113 @@ segmentation_detections(struct noisechiselparams *p)
/***********************************************************************/
+/***************** Output *****************/
+/***********************************************************************/
+void
+segment_output(struct segmentparams *p)
+{
+ gal_fits_list_key_t *keys=NULL;
+
+ /* The Sky-subtracted input (if requested). */
+ if(!p->rawoutput)
+ gal_fits_img_write(p->input, p->cp.output, NULL, PROGRAM_NAME);
+
+
+ /* The clump labels. */
+ gal_fits_key_list_add(&keys, GAL_TYPE_FLOAT32, "CLUMPSN", 0,
+ &p->clumpsnthresh, 0, "Minimum S/N of true clumps",
+ 0, "ratio");
+ gal_fits_key_list_add(&keys, GAL_TYPE_SIZE_T, "NUMLABS", 0,
+ &p->numclumps, 0, "Total number of clumps", 0,
+ "counter");
+ p->clabel->name="CLUMPS";
+ gal_fits_img_write(p->clabel, p->cp.output, keys, PROGRAM_NAME);
+ p->clabel->name=NULL;
+ keys=NULL;
+
+
+ /* The object labels. */
+ gal_fits_key_list_add(&keys, GAL_TYPE_SIZE_T, "NUMLABS", 0,
+ &p->numobjects, 0, "Total number of objects", 0,
+ "counter");
+ p->olabel->name="OBJECTS";
+ gal_fits_img_write(p->olabel, p->cp.output, keys, PROGRAM_NAME);
+ p->olabel->name=NULL;
+ keys=NULL;
+
+
+ /* The Standard deviation image. */
+ if(!p->rawoutput)
+ {
+ /* See if any keywords should be written (possibly inherited from the
+ detection program). */
+ if( !isnan(p->maxstd) )
+ gal_fits_key_list_add(&keys, GAL_TYPE_FLOAT32, "MAXSTD", 0,
+ &p->maxstd, 0,
+ "Maximum raw tile standard deviation", 0,
+ p->input->unit);
+ if( !isnan(p->minstd) )
+ gal_fits_key_list_add(&keys, GAL_TYPE_FLOAT32, "MINSTD", 0,
+ &p->minstd, 0,
+ "Minimum raw tile standard deviation", 0,
+ p->input->unit);
+ if( !isnan(p->medstd) )
+ gal_fits_key_list_add(&keys, GAL_TYPE_FLOAT32, "MEDSTD", 0,
+ &p->medstd, 0,
+ "Median raw tile standard deviation", 0,
+ p->input->unit);
+
+ /* Write the STD dataset into the output file. */
+ p->std->name="SKY_STD";
+ if(p->std->size == p->input->size)
+ gal_fits_img_write(p->std, p->cp.output, keys, PROGRAM_NAME);
+ else
+ gal_tile_full_values_write(p->std, &p->cp.tl, 1, p->cp.output, keys,
+ PROGRAM_NAME);
+ p->std->name=NULL;
+ }
+}
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+/***********************************************************************/
/***************** High level function *****************/
/***********************************************************************/
void
-segmentation(struct noisechiselparams *p)
+segment(struct segmentparams *p)
{
float *f;
char *msg;
- int32_t *l, *lf;
+ int32_t *c, *cf;
struct timeval t1;
- /* To keep the user up to date. */
- if(!p->cp.quiet)
- {
- if(!p->cp.quiet) gettimeofday(&t1, NULL);
- gal_timing_report(NULL, "Starting segmentation.",
- 1);
- }
+ /* Get starting time for later reporting if necessary. */
+ if(!p->cp.quiet) gettimeofday(&t1, NULL);
+
+
+ /* Prepare the inputs. */
+ segment_convolve(p);
+ segment_initialize(p);
- /* If a check segmentation image was requested, then put in the
- inputs. */
+ /* If a check segmentation image was requested, then start filling it
+ in. */
if(p->segmentationname)
{
gal_fits_img_write(p->input, p->segmentationname, NULL, PROGRAM_NAME);
@@ -882,31 +1130,33 @@ segmentation(struct noisechiselparams *p)
PROGRAM_NAME);
p->olabel->name=NULL;
}
-
-
- /* Allocate the clump labels image. */
- p->clabel=gal_data_alloc(NULL, p->olabel->type, p->olabel->ndim,
- p->olabel->dsize, p->olabel->wcs, 1,
- p->cp.minmapsize, NULL, NULL, NULL);
- p->clabel->flag=p->input->flag;
-
-
- /* Set any possibly existing NaN values to blank. */
- f=p->input->array; lf=(l=p->clabel->array)+p->clabel->size;
- do if(isnan(*f++)) *l=GAL_BLANK_INT32; while(++l<lf);
+ if(!p->cp.quiet)
+ printf(" - Input number of detections: %zu\n", p->numdetections);
/* Find the clump S/N threshold. */
- clumps_true_find_sn_thresh(p);
+ if( isnan(p->clumpsnthresh) )
+ {
+ gal_timing_report(NULL, "Finding true clumps...", 1);
+ clumps_true_find_sn_thresh(p);
+ }
+ else
+ {
+ if( asprintf(&msg, "Given S/N for true clumps: %g",
+ p->clumpsnthresh) <0 )
+ error(EXIT_FAILURE, 0, "%s: asprintf allocation", __func__);
+ gal_timing_report(NULL, msg, 1);
+ free(msg);
+ }
/* Reset the clabel array to find true clumps in objects. */
- f=p->input->array; lf=(l=p->clabel->array)+p->clabel->size;
- do *l = isnan(*f++) ? GAL_BLANK_INT32 : 0; while(++l<lf);
+ f=p->input->array; cf=(c=p->clabel->array)+p->clabel->size;
+ do *c = isnan(*f++) ? GAL_BLANK_INT32 : 0; while(++c<cf);
/* Find true clumps over the detected regions. */
- segmentation_detections(p);
+ segment_detections(p);
/* Report the results and timing to the user. */
@@ -926,4 +1176,8 @@ segmentation(struct noisechiselparams *p)
if(p->segmentationname && !p->continueaftercheck)
ui_abort_after_check(p, p->segmentationname, NULL,
"showing all segmentation steps");
+
+
+ /* Write the output. */
+ segment_output(p);
}
diff --git a/bin/noisechisel/segmentation.h b/bin/segment/segment.h
similarity index 80%
rename from bin/noisechisel/segmentation.h
rename to bin/segment/segment.h
index 6e17401..2e73e06 100644
--- a/bin/noisechisel/segmentation.h
+++ b/bin/segment/segment.h
@@ -1,6 +1,6 @@
/*********************************************************************
-NoiseChisel - Detect and segment signal in a noisy dataset.
-NoiseChisel is part of GNU Astronomy Utilities (Gnuastro) package.
+Segment - Segment initial labels based on signal structure.
+Segment is part of GNU Astronomy Utilities (Gnuastro) package.
Original author:
Mohammad Akhlaghi <address@hidden>
@@ -20,10 +20,10 @@ General Public License for more details.
You should have received a copy of the GNU General Public License
along with Gnuastro. If not, see <http://www.gnu.org/licenses/>.
**********************************************************************/
-#ifndef SEGMENTATION_H
-#define SEGMENTATION_H
+#ifndef SEGMENT_H
+#define SEGMENT_H
void
-segmentation(struct noisechiselparams *p);
+segment(struct segmentparams *p);
#endif
diff --git a/bin/noisechisel/ui.c b/bin/segment/ui.c
similarity index 60%
copy from bin/noisechisel/ui.c
copy to bin/segment/ui.c
index 87cd718..dd601fd 100644
--- a/bin/noisechisel/ui.c
+++ b/bin/segment/ui.c
@@ -1,11 +1,11 @@
/*********************************************************************
-NoiseChisel - Detect and segment signal in a noisy dataset.
-NoiseChisel is part of GNU Astronomy Utilities (Gnuastro) package.
+Segment - Segment initial labels based on signal structure.
+Segment is part of GNU Astronomy Utilities (Gnuastro) package.
Original author:
Mohammad Akhlaghi <address@hidden>
Contributing author(s):
-Copyright (C) 2015-2018, Free Software Foundation, Inc.
+Copyright (C) 2018, Free Software Foundation, Inc.
Gnuastro is free software: you can redistribute it and/or modify it
under the terms of the GNU General Public License as published by the
@@ -30,7 +30,6 @@ along with Gnuastro. If not, see
<http://www.gnu.org/licenses/>.
#include <gnuastro/wcs.h>
#include <gnuastro/fits.h>
-#include <gnuastro/blank.h>
#include <gnuastro/threads.h>
#include <gnuastro/dimension.h>
@@ -64,10 +63,10 @@ static char
args_doc[] = "ASTRdata";
const char
-doc[] = GAL_STRINGS_TOP_HELP_INFO PROGRAM_NAME" Detects and segments signal "
- "that is deeply burried in noise. It employs a noise-based detection and "
- "segmentation method enabling it to be very resilient to the rich diversity "
- "of shapes in astronomical targets.\n"
+doc[] = GAL_STRINGS_TOP_HELP_INFO PROGRAM_NAME" will segment an initially "
+ "labeled region based on structure with the signal. It will first find "
+ "true clumps (local maxima), estimate which ones have strong connections, "
+ "and then grow them to cover the full area of each detection.\n"
GAL_STRINGS_MORE_HELP_INFO
/* After the list of options: */
"\v"
@@ -96,7 +95,7 @@ doc[] = GAL_STRINGS_TOP_HELP_INFO PROGRAM_NAME" Detects and
segments signal "
/********* Initialize & Parse command-line **************/
/**************************************************************/
static void
-ui_initialize_options(struct noisechiselparams *p,
+ui_initialize_options(struct segmentparams *p,
struct argp_option *program_options,
struct argp_option *gal_commonopts_options)
{
@@ -113,6 +112,11 @@ ui_initialize_options(struct noisechiselparams *p,
cp->numthreads = gal_threads_number();
cp->coptions = gal_commonopts_options;
+ p->medstd = NAN;
+ p->minstd = NAN;
+ p->maxstd = NAN;
+ p->clumpsnthresh = NAN;
+
/* Modify common options. */
for(i=0; !gal_options_is_last(&cp->coptions[i]); ++i)
{
@@ -147,10 +151,10 @@ ui_initialize_options(struct noisechiselparams *p,
/* Parse a single option: */
-static error_t
+error_t
parse_opt(int key, char *arg, struct argp_state *state)
{
- struct noisechiselparams *p = state->input;
+ struct segmentparams *p = state->input;
/* Pass `gal_options_common_params' into the child parser. */
state->child_inputs[0] = &p->cp;
@@ -213,37 +217,20 @@ parse_opt(int key, char *arg, struct argp_state *state)
/* Read and check ONLY the options. When arguments are involved, do the
check in `ui_check_options_and_arguments'. */
static void
-ui_read_check_only_options(struct noisechiselparams *p)
+ui_read_check_only_options(struct segmentparams *p)
{
- /* If the convolved option is given, then the convolved HDU is also
- mandatory. */
- if(p->convolvedname && p->convolvedhdu==NULL)
+ /* If the convolved HDU is given. */
+ if(p->convolvedname && p->chdu==NULL)
error(EXIT_FAILURE, 0, "no value given to `--convolvedhdu'. When the "
"`--convolved' option is called (to specify a convolved image "
"and avoid convolution) it is mandatory to also specify a HDU "
"for it");
- /* Make sure the connectivities have the correct values. */
- if(p->erodengb!=4 && p->erodengb!=8)
- error(EXIT_FAILURE, 0, "%zu not acceptable for `--erodengb'. It must "
- "be 4 or 8 (specifying the type of connectivity)", p->erodengb);
- if(p->openingngb!=4 && p->openingngb!=8)
- error(EXIT_FAILURE, 0, "%zu not acceptable for `--openingngb'. It must "
- "be 4 or 8 (specifying the type of connectivity)", p->openingngb);
-
- /* Make sure that the no-erode-quantile is not smaller or equal to
- qthresh. */
- if( p->noerodequant <= p->qthresh)
- error(EXIT_FAILURE, 0, "the quantile for no erosion (`--noerodequant') "
- "must be larger than the base quantile threshold (`--qthresh', "
- "or `-t'). You have provided %.4f and %.4f for the former and "
- "latter, respectively", p->noerodequant, p->qthresh);
-
- /* For the options that make tables, the table formation option is
+ /* For the options that make tables, the table format option is
mandatory. */
- if( (p->checkdetsn || p->checkclumpsn) && p->cp.tableformat==0 )
+ if( p->checksn && p->cp.tableformat==0 )
error(EXIT_FAILURE, 0, "`--tableformat' is necessary with the "
- "`--checkdetsn' and `--checkclumpsn' options.\n"
+ "`--checksn' option.\n"
"Please see description for `--tableformat' after running the "
"following command for more information (use `SPACE' to go down "
"the page and `q' to return to the command-line):\n\n"
@@ -263,21 +250,6 @@ ui_read_check_only_options(struct noisechiselparams *p)
"(starting from zero), extension name, or anything "
"acceptable by CFITSIO");
}
-
- /* Wide kernel checks. */
- if(p->widekernelname)
- {
- /* Check if it exists. */
- gal_checkset_check_file(p->widekernelname);
-
- /* If its FITS, see if a HDU has been provided. */
- if( gal_fits_name_is_fits(p->widekernelname) && p->wkhdu==NULL )
- error(EXIT_FAILURE, 0, "no HDU specified for wide kernel. When the "
- "wide kernel is a FITS file, a HDU must also be specified. You "
- "can use the `--khdu' option and give it the HDU number "
- "(starting from zero), extension name, or anything "
- "acceptable by CFITSIO");
- }
}
@@ -285,21 +257,22 @@ ui_read_check_only_options(struct noisechiselparams *p)
static void
-ui_check_options_and_arguments(struct noisechiselparams *p)
+ui_check_options_and_arguments(struct segmentparams *p)
{
- /* Basic input file checks. */
+ /* Make sure an input file name was given and if it was a FITS file, that
+ a HDU is also given. */
if(p->inputname)
{
/* Check if it exists. */
gal_checkset_check_file(p->inputname);
- /* If its FITS, see if a HDU has been provided. */
+ /* If it is FITS, a HDU is also mandatory. */
if( gal_fits_name_is_fits(p->inputname) && p->cp.hdu==NULL )
- error(EXIT_FAILURE, 0, "no HDU specified for input. When the input "
- "is a FITS file, a HDU must also be specified, you can use "
- "the `--hdu' (`-h') option and give it the HDU number "
- "(starting from zero), extension name, or anything "
- "acceptable by CFITSIO");
+ error(EXIT_FAILURE, 0, "no HDU specified. When the input is a FITS "
+ "file, a HDU must also be specified, you can use the `--hdu' "
+ "(`-h') option and give it the HDU number (starting from "
+ "zero), extension name, or anything acceptable by CFITSIO");
+
}
else
error(EXIT_FAILURE, 0, "no input file is specified");
@@ -328,7 +301,23 @@ ui_check_options_and_arguments(struct noisechiselparams *p)
/*************** Preparations *******************/
/**************************************************************/
static void
-ui_set_output_names(struct noisechiselparams *p)
+ui_set_used_names(struct segmentparams *p)
+{
+ p->useddetectionname = p->detectionname ? p->detectionname : p->inputname;
+
+ p->usedstdname = ( p->stdname
+ ? p->stdname
+ : ( p->detectionname
+ ? p->detectionname
+ : p->inputname ) );
+}
+
+
+
+
+
+static void
+ui_set_output_names(struct segmentparams *p)
{
char *output=p->cp.output;
char *basename = output ? output : p->inputname;
@@ -341,53 +330,20 @@ ui_set_output_names(struct noisechiselparams *p)
/* When the output name is given (possibly with directory
information), the check images will also be put in that same
- directory.. */
+ directory. */
p->cp.keepinputdir=1;
}
else
p->cp.output=gal_checkset_automatic_output(&p->cp, p->inputname,
- "_labeled.fits");
+ "_segmented.fits");
/* Tile check. */
if(p->cp.tl.checktiles)
p->cp.tl.tilecheckname=gal_checkset_automatic_output(&p->cp, basename,
"_tiles.fits");
- /* Quantile threshold. */
- if(p->checkqthresh)
- p->qthreshname=gal_checkset_automatic_output(&p->cp, basename,
- "_qthresh.fits");
-
- /* Initial detection Sky values. */
- if(p->checkdetsky)
- p->detskyname=gal_checkset_automatic_output(&p->cp, basename,
- "_detsky.fits");
-
- /* Pseudo-detection S/N values. */
- if(p->checkdetsn)
- {
- p->detsn_s_name=gal_checkset_automatic_output(&p->cp, basename,
- ( p->cp.tableformat==GAL_TABLE_FORMAT_TXT
- ? "_detsn_sky.txt" : "_detsn_sky.fits") );
- p->detsn_d_name=gal_checkset_automatic_output(&p->cp, basename,
- ( p->cp.tableformat==GAL_TABLE_FORMAT_TXT
- ? "_detsn_det.txt" : "_detsn_det.fits") );
- p->detsn_D_name=gal_checkset_automatic_output(&p->cp, basename,
- ( p->cp.tableformat==GAL_TABLE_FORMAT_TXT
- ? "_detsn_grown.txt" : "_detsn_grown.fits") );
- }
-
- /* Detection steps. */
- if(p->checkdetection)
- p->detectionname=gal_checkset_automatic_output(&p->cp, basename,
- "_det.fits");
-
- /* Detection steps. */
- if(p->checksky)
- p->skyname=gal_checkset_automatic_output(&p->cp, basename, "_sky.fits");
-
/* Clump S/N values. */
- if(p->checkclumpsn)
+ if(p->checksn)
{
p->clumpsn_s_name=gal_checkset_automatic_output(&p->cp, basename,
( p->cp.tableformat==GAL_TABLE_FORMAT_TXT
@@ -400,7 +356,7 @@ ui_set_output_names(struct noisechiselparams *p)
/* Segmentation steps. */
if(p->checksegmentation)
p->segmentationname=gal_checkset_automatic_output(&p->cp, basename,
- "_seg.fits");
+ "_segcheck.fits");
}
@@ -408,60 +364,78 @@ ui_set_output_names(struct noisechiselparams *p)
static void
-ui_prepare_kernel(struct noisechiselparams *p)
+ui_prepare_inputs(struct segmentparams *p)
{
- /* The default kernel. It was created by saving the following commands in a
- script and running it. It will create a plain text array along with a
- FITS image. The crop is because the first and last rows and columns of
- all PSFs made by MakeProfiles are blank (zero) when you run with
- oversample=1. You can keep the spaces when copying and pasting ;-). Just
- make it executable and run it.
-
- set -o errexit # Stop if a program returns false.
- echo "0 0.0 0.0 3 2 0 0 1 1 5" > tmp.txt
- export GSL_RNG_TYPE=ranlxs2
- export GSL_RNG_SEED=1
- astmkprof tmp.txt --oversample=1 --envseed --numrandom=10000 \
- --tolerance=0.01 --nomerged
- astcrop 0_tmp.fits --section=2:*-1,2:*-1 --zeroisnotblank \
- --output=fwhm2.fits
- astconvertt fwhm2.fits --output=fwhm2.txt
- rm 0_tmp.fits tmp.txt
- */
- size_t kernel_2d_dsize[2]={11,11};
- float *f, *ff, *k, kernel_2d[121]=
- {
- 0, 0, 0, 0, 0, 6.57699e-09, 0, 0, 0, 0, 0,
+ /* Read the input as a single precision floating point dataset. */
+ p->input = gal_fits_img_read_to_type(p->inputname, p->cp.hdu,
+ GAL_TYPE_FLOAT32, p->cp.minmapsize);
+ p->input->wcs = gal_wcs_read(p->inputname, p->cp.hdu, 0, 0,
+ &p->input->nwcs);
+ if(p->input->name) free(p->input->name);
+ gal_checkset_allocate_copy("INPUT", &p->input->name);
- 0, 0, 6.57699e-09, 2.10464e-07, 1.68371e-06, 3.36742e-06, 1.68371e-06,
- 2.10464e-07, 6.57699e-09, 0, 0,
- 0, 6.57699e-09, 8.41855e-07, 2.69394e-05, 0.000383569, 0.000717224,
- 0.000379782, 2.69394e-05, 8.41855e-07, 6.57699e-09, 0,
+ /* Check for blank values to help later processing. */
+ gal_blank_present(p->input, 1);
- 0, 2.10464e-07, 2.69394e-05, 0.00140714, 0.00888549, 0.016448,
- 0.00867408, 0.00138203, 2.69394e-05, 2.10464e-07, 0,
- 0, 1.68371e-06, 0.000381138, 0.00875434, 0.0573377, 0.106308, 0.0570693,
- 0.00891745, 0.000378914, 1.68371e-06, 0,
+ /* Segment currently only works on 2D datasets (images). */
+ if(p->input->ndim!=2)
+ error(EXIT_FAILURE, 0, "%s (hdu: %s) has %zu dimensions but Segment "
+ "can only operate on 2D datasets (images)", p->inputname, p->cp.hdu,
+ p->input->ndim);
+
+
+ /* If a convolved image is given, read it. */
+ if(p->convolvedname)
+ {
+ /* Read the input convolved image. */
+ p->conv = gal_fits_img_read_to_type(p->convolvedname, p->chdu,
+ GAL_TYPE_FLOAT32, p->cp.minmapsize);
+ p->conv->wcs=gal_wcs_copy(p->input->wcs);
+
+ /* Make sure it is the same size as the input. */
+ if( gal_data_dsize_is_different(p->input, p->conv) )
+ error(EXIT_FAILURE, 0, "%s (hdu %s), given to `--convolved' and "
+ "`--chdu', is not the same size as the input (%s, hdu: %s)",
+ p->convolvedname, p->chdu, p->inputname, p->cp.hdu);
+ }
+
+
+ /* Read the detected label image and check its type and size. */
+ p->olabel = gal_fits_img_read(p->useddetectionname, p->dhdu,
+ p->cp.minmapsize);
+ if( gal_data_dsize_is_different(p->input, p->olabel) )
+ error(EXIT_FAILURE, 0, "`%s' (hdu: %s) and `%s' (hdu: %s) have a"
+ "different dimension/size", p->useddetectionname, p->dhdu,
+ p->inputname, p->cp.hdu);
+ if(p->olabel->type==GAL_TYPE_FLOAT32 || p->olabel->type==GAL_TYPE_FLOAT64)
+ error(EXIT_FAILURE, 0, "%s (hdu: %s) has a `%s' type. The detection "
+ "(labeled) map must have an integer type (labels/classes can only "
+ "be integers). If the pixel values are integers, but only the "
+ "numerical type of the image is floating-point, you can use the "
+ "command below to convert it to a 32-bit (signed) integer type:\n\n"
+ " $ astarithmetic %s int32 -h%s\n\n", p->useddetectionname,
+ p->dhdu, gal_type_name(p->olabel->type, 1), p->useddetectionname,
+ p->dhdu);
+ p->olabel = gal_data_copy_to_new_type_free(p->olabel, GAL_TYPE_INT32);
+ p->olabel->wcs=gal_wcs_copy(p->input->wcs);
+}
- 6.57699e-09, 3.36742e-06, 0.00071364, 0.0164971, 0.106865, 0.197316,
- 0.106787, 0.0166434, 0.000713827, 3.36742e-06, 6.57699e-09,
- 0, 1.68371e-06, 0.000215515, 0.00894112, 0.0573699, 0.106239, 0.0567907,
- 0.00901191, 0.000215515, 1.68371e-06, 0,
- 0, 2.10464e-07, 2.69394e-05, 0.00135085, 0.0089288, 0.0164171,
- 0.00879334, 0.0013622, 2.69394e-05, 2.10464e-07, 0,
- 0, 6.57699e-09, 8.41855e-07, 2.69394e-05, 0.000215515, 0.000724137,
- 0.000215515, 2.69394e-05, 8.41855e-07, 6.57699e-09, 0,
- 0, 0, 6.57699e-09, 2.10464e-07, 1.68371e-06, 3.36742e-06, 1.68371e-06,
- 2.10464e-07, 6.57699e-09, 0, 0,
+/* Prepare the input kernel. */
+static void
+ui_prepare_kernel(struct segmentparams *p)
+{
+ float *f, *ff, *k;
- 0, 0, 0, 0, 0, 6.57699e-09, 0, 0, 0, 0, 0
- };
+/* Since the default kernel has to be identical between NoiseChisel and
+ Segment, we have defined it in a shared header file to be accessible by
+ both programs. */
+#include <gnuastro-internal/kernel-2d.h>
/* If a kernel file is given, then use it. Otherwise, use the default
kernel. */
@@ -486,40 +460,25 @@ ui_prepare_kernel(struct noisechiselparams *p)
ff=(f=kernel_2d)+gal_dimension_total_size(2, p->kernel->dsize);
do *k++=*f; while(++f<ff);
}
-
-
- /* If a wide kernel is given, then read it into memory. Otherwise, just
- ignore it. */
- if(p->widekernelname)
- p->widekernel=gal_fits_img_read_kernel(p->widekernelname, p->wkhdu,
- p->cp.minmapsize);
}
+/* Set up the tessellation. */
static void
-ui_prepare_tiles(struct noisechiselparams *p)
+ui_prepare_tiles(struct segmentparams *p)
{
gal_data_t *check;
struct gal_tile_two_layer_params *tl=&p->cp.tl, *ltl=&p->ltl;
/* Check the tile parameters for the small tile sizes and make the tile
- structure. We will also need the dimensions of the tile with the
- maximum required memory. */
- p->maxtsize=gal_data_malloc_array(GAL_TYPE_SIZE_T, p->input->ndim,
- __func__, "p->maxtsize");
+ structure. */
gal_tile_full_sanity_check(p->inputname, p->cp.hdu, p->input, tl);
gal_tile_full_two_layers(p->input, tl);
gal_tile_full_permutation(tl);
- for(check=tl->tiles; check!=NULL; check=check->next)
- if( check->size > p->maxtcontig )/* p->maxtcontig initialized to 0. */
- {
- p->maxtcontig=check->size;
- memcpy(p->maxtsize, check->dsize, tl->ndim*sizeof *p->maxtsize);
- }
/* Make the large tessellation, except for the size, the rest of the
@@ -529,20 +488,12 @@ ui_prepare_tiles(struct noisechiselparams *p)
ltl->workoverch = tl->workoverch;
ltl->checktiles = tl->checktiles;
ltl->oneelempertile = tl->oneelempertile;
- p->maxltsize=gal_data_malloc_array(GAL_TYPE_SIZE_T, p->input->ndim,
- __func__, "p->maxltsize");
gal_tile_full_sanity_check(p->inputname, p->cp.hdu, p->input, ltl);
gal_tile_full_two_layers(p->input, ltl);
gal_tile_full_permutation(ltl);
- for(check=ltl->tiles; check!=NULL; check=check->next)
- if( check->size > p->maxltcontig )/* p->maxltcontig initialized to 0. */
- {
- p->maxltcontig=check->size;
- memcpy(p->maxltsize, check->dsize, ltl->ndim*sizeof *p->maxltsize);
- }
- /* If the input has blank elements, then set teh appropriate flag for
+ /* If the input has blank elements, then set the appropriate flag for
each tile.*/
if( p->input->flag & GAL_DATA_FLAG_HASBLANK )
{
@@ -571,6 +522,7 @@ ui_prepare_tiles(struct noisechiselparams *p)
/* Free the name. */
free(tl->tilecheckname);
+ tl->tilecheckname=NULL;
}
}
@@ -579,63 +531,167 @@ ui_prepare_tiles(struct noisechiselparams *p)
static void
-ui_preparations(struct noisechiselparams *p)
+ui_check_size(gal_data_t *base, gal_data_t *comp, size_t numtiles,
+ char *bname, char *bhdu, char *cname, char *chdu)
{
- /* Prepare the names of the outputs. */
- ui_set_output_names(p);
+ if( gal_data_dsize_is_different(base, comp) && numtiles!=comp->size)
+ error(EXIT_FAILURE, 0, "%s (hdu: %s): doesn't have the right size "
+ "(%zu elements or pixels).\n\n"
+ "It must either be the same size as `%s' (hdu: `%s'), or "
+ "it must have the same number of elements as the total "
+ "number of tiles in the tessellation (%zu). In the latter "
+ "case, each pixel is assumed to be a fixed value for a "
+ "complete tile.\n\n"
+ "Run with `-P' to see the (tessellation) options/settings "
+ "and their values). For more information on tessellation in "
+ "Gnuastro, please run the following command (use the arrow "
+ "keys for up and down and press `q' to return to the "
+ "command-line):\n\n"
+ " $ info gnuastro tessellation",
+ cname, chdu, comp->size, bname, bhdu, numtiles);
+}
- /* Read the input as a single precision floating point dataset. */
- p->input = gal_fits_img_read_to_type(p->inputname, p->cp.hdu,
- GAL_TYPE_FLOAT32, p->cp.minmapsize);
- p->input->wcs = gal_wcs_read(p->inputname, p->cp.hdu, 0, 0,
- &p->input->nwcs);
- if(p->input->name==NULL)
- gal_checkset_allocate_copy("INPUT", &p->input->name);
- /* NoiseChisel currently only works on 2D datasets (images). */
- if(p->input->ndim!=2)
- error(EXIT_FAILURE, 0, "%s (hdu: %s) has %zu dimensions but NoiseChisel "
- "can only operate on 2D datasets (images)", p->inputname, p->cp.hdu,
- p->input->ndim);
+/* Subtract `sky' from the input dataset depending on its size (it may be
+ the whole array or a tile-values array).. */
+static void
+ui_subtract_sky(gal_data_t *in, gal_data_t *sky,
+ struct gal_tile_two_layer_params *tl)
+{
+ size_t tid;
+ gal_data_t *tile;
+ float *f, *s, *sf, *skyarr=sky->array;
- /* If a convolved image was given, read it in. Otherwise, read the given
- kernel. */
- if(p->convolvedname)
+ /* It is the same size as the input. */
+ if( gal_data_dsize_is_different(in, sky)==0 )
{
- /* Read the input convolved image. */
- p->conv = gal_fits_img_read_to_type(p->convolvedname, p->convolvedhdu,
- GAL_TYPE_FLOAT32, p->cp.minmapsize);
+ f=in->array;
+ sf=(s=sky->array)+sky->size;
+ do *f++-=*s; while(++s<sf);
+ }
- /* Make sure the convolved image is the same size as the input. */
- if( gal_data_dsize_is_different(p->input, p->conv) )
- error(EXIT_FAILURE, 0, "%s (hdu %s), given to `--convolved' and "
- "`--convolvehdu', is not the same size as NoiseChisel's "
- "input: %s (hdu: %s)", p->convolvedname, p->convolvedhdu,
- p->inputname, p->cp.hdu);
+ /* It is the same size as the number of tiles. */
+ else if( tl->tottiles==sky->size )
+ {
+ /* Go over all the tiles. */
+ for(tid=0; tid<tl->tottiles; ++tid)
+ {
+ /* For easy reading. */
+ tile=&tl->tiles[tid];
+
+ /* Subtract the Sky value from the input image. */
+ GAL_TILE_PARSE_OPERATE(tile, NULL, 0, 0, {*i-=skyarr[tid];});
+ }
}
+
+ /* The size must have been checked before, so if control reaches here, we
+ have a bug! */
else
- ui_prepare_kernel(p);
+ error(EXIT_FAILURE, 0, "%s: a bug! Please contact us at %s to fix "
+ "the problem. For some reason, the size doesn't match", __func__,
+ PACKAGE_BUGREPORT);
+}
- /* Check for blank values to help later processing. */
- gal_blank_present(p->input, 1);
+
+/* The Sky and Sky standard deviation images can be a `oneelempertile'
+ image (only one element/pixel for a tile). So we need to do some extra
+ checks on them (after reading the tessellation). */
+static void
+ui_read_std(struct segmentparams *p)
+{
+ struct gal_tile_two_layer_params *tl=&p->cp.tl;
+ gal_data_t *sky, *keys=gal_data_array_calloc(3);
+
+ /* The standard devitaion image. */
+ p->std=gal_fits_img_read_to_type(p->usedstdname, p->stdhdu,
+ GAL_TYPE_FLOAT32, p->cp.minmapsize);
+ ui_check_size(p->input, p->std, tl->tottiles, p->inputname, p->cp.hdu,
+ p->usedstdname, p->stdhdu);
+
+
+ /* If a Sky image is given, subtract it from the values and convolved
+ images, then free it. */
+ if(p->skyname)
+ {
+ /* Read the Sky dataset. */
+ sky=gal_fits_img_read_to_type(p->skyname, p->skyhdu, GAL_TYPE_FLOAT32,
+ p->cp.minmapsize);
+
+ /* Check its size. */
+ ui_check_size(p->input, sky, tl->tottiles, p->inputname, p->cp.hdu,
+ p->skyname, p->skyhdu);
+
+ /* Subtract it from the input. */
+ ui_subtract_sky(p->input, sky, tl);
+
+ /* If a convolved image is given, subtract the Sky from that too. */
+ if(p->conv)
+ ui_subtract_sky(p->conv, sky, tl);
+
+ /* Clean up. */
+ gal_data_free(sky);
+ }
+
+
+ /* When the Standard deviation image is made by NoiseChisel, it puts
+ three basic statistics of the pre-interpolation distribution of
+ standard deviations in `MEDSTD', `MINSTD' and `MAXSTD'. The `MEDSTD'
+ in particular is most important because it can't be inferred after the
+ interpolations and it can be useful in MakeCatalog later to give a
+ more accurate estimate of the noise level. So if they are present, we
+ will read them here and write them to the STD output (which is created
+ when `--rawoutput' is not given). */
+ if(!p->rawoutput)
+ {
+ keys[0].next=&keys[1];
+ keys[1].next=&keys[2];
+ keys[2].next=NULL;
+ keys[0].array=&p->medstd; keys[0].name="MEDSTD";
+ keys[1].array=&p->minstd; keys[1].name="MINSTD";
+ keys[2].array=&p->maxstd; keys[2].name="MAXSTD";
+ keys[0].type=keys[1].type=keys[2].type=GAL_TYPE_FLOAT32;
+ gal_fits_key_read(p->usedstdname, p->stdhdu, keys, 0, 0);
+ if(keys[0].status) p->medstd=NAN;
+ if(keys[1].status) p->minstd=NAN;
+ if(keys[2].status) p->maxstd=NAN;
+ keys[0].name=keys[1].name=keys[2].name=NULL;
+ keys[0].array=keys[1].array=keys[2].array=NULL;
+ gal_data_array_free(keys, 3, 1);
+ }
+}
+
+
+
+
+
+static void
+ui_preparations(struct segmentparams *p)
+{
+ /* Set the input names. */
+ ui_set_used_names(p);
+
+ /* Prepare the names of the outputs. */
+ ui_set_output_names(p);
+
+ /* Read the input datasets. */
+ ui_prepare_inputs(p);
+
+ /* If a convolved image was given, read it in. Otherwise, read the given
+ kernel. */
+ if(p->conv==NULL)
+ ui_prepare_kernel(p);
+
/* Prepare the tessellation. */
ui_prepare_tiles(p);
-
- /* Allocate space for the over-all necessary arrays. */
- p->binary=gal_data_alloc(NULL, GAL_TYPE_UINT8, p->input->ndim,
- p->input->dsize, p->input->wcs, 0,
- p->cp.minmapsize, NULL, "binary", NULL);
- p->olabel=gal_data_alloc(NULL, GAL_TYPE_INT32, p->input->ndim,
- p->input->dsize, p->input->wcs, 0,
- p->cp.minmapsize, NULL, "labels", NULL);
- p->binary->flag = p->olabel->flag = p->input->flag;
+ /* Prepare the (optional Sky, and) Sky Standard deviation image. */
+ ui_read_std(p);
}
@@ -657,11 +713,10 @@ ui_preparations(struct noisechiselparams *p)
/**************************************************************/
-/************ High level reading function *************/
+/************ Set the parameters *************/
/**************************************************************/
void
-ui_read_check_inputs_setup(int argc, char *argv[],
- struct noisechiselparams *p)
+ui_read_check_inputs_setup(int argc, char *argv[], struct segmentparams *p)
{
struct gal_options_common_params *cp=&p->cp;
@@ -719,27 +774,25 @@ ui_read_check_inputs_setup(int argc, char *argv[],
printf(" - Using %zu CPU thread%s\n", p->cp.numthreads,
p->cp.numthreads==1 ? "." : "s.");
printf(" - Input: %s (hdu: %s)\n", p->inputname, p->cp.hdu);
+ if(p->skyname)
+ printf(" - Sky: %s (hdu: %s)\n", p->skyname, p->skyhdu);
if(p->convolvedname)
- printf(" - Convolved input: %s (hdu: %s)\n",
- p->convolvedname, p->convolvedhdu);
+ printf(" - Convolved input: %s (hdu: %s)\n", p->convolvedname,
+ p->chdu);
else
{
if(p->kernelname)
{
if( strcmp(p->kernelname, UI_NO_CONV_KERNEL_NAME) )
- printf(" - %s: %s (hdu: %s)\n",
- p->widekernelname ? "Sharp Kernel" : "Kernel",
- p->kernelname, p->khdu);
+ printf(" - Kernel: %s (hdu: %s)\n", p->kernelname, p->khdu);
else
printf(" - No convolution requested.\n");
}
else
- printf(" - %s: FWHM=2 pixel Gaussian.\n",
- p->widekernelname ? "Sharp Kernel" : "Kernel");
+ printf(" - Kernel: FWHM=2 pixel Gaussian.\n");
}
- if(p->widekernelname)
- printf(" - Wide Kernel: %s (hdu: %s)\n", p->widekernelname,
- p->wkhdu);
+ printf(" - Detection: %s (hdu: %s)\n", p->useddetectionname, p->dhdu);
+ printf(" - Sky STD: %s (hdu: %s)\n", p->usedstdname, p->stdhdu);
}
}
@@ -763,10 +816,10 @@ ui_read_check_inputs_setup(int argc, char *argv[],
/**************************************************************/
-/************ Pre-finish/abort operations *************/
+/************ Free allocated, report *************/
/**************************************************************/
void
-ui_abort_after_check(struct noisechiselparams *p, char *filename,
+ui_abort_after_check(struct segmentparams *p, char *filename,
char *file2name, char *description)
{
char *name;
@@ -807,39 +860,29 @@ ui_abort_after_check(struct noisechiselparams *p, char
*filename,
void
-ui_free_report(struct noisechiselparams *p, struct timeval *t1)
+ui_free_report(struct segmentparams *p, struct timeval *t1)
{
- /* Free the simply allocated spaces. */
+ /* Free the allocated arrays: */
free(p->cp.hdu);
- free(p->maxtsize);
- free(p->maxltsize);
free(p->cp.output);
- if(p->skyname) free(p->skyname);
- if(p->detskyname) free(p->detskyname);
- if(p->qthreshname) free(p->qthreshname);
- if(p->detsn_s_name) free(p->detsn_s_name);
- if(p->detsn_d_name) free(p->detsn_d_name);
- if(p->detectionname) free(p->detectionname);
- if(p->clumpsn_s_name) free(p->clumpsn_s_name);
- if(p->clumpsn_d_name) free(p->clumpsn_d_name);
- if(p->segmentationname) free(p->segmentationname);
-
- /* Free the allocated datasets. */
- gal_data_free(p->sky);
- gal_data_free(p->std);
- gal_data_free(p->wconv);
gal_data_free(p->input);
gal_data_free(p->kernel);
gal_data_free(p->binary);
gal_data_free(p->olabel);
gal_data_free(p->clabel);
- gal_data_free(p->widekernel);
+ if(p->khdu) free(p->khdu);
+ if(p->chdu) free(p->chdu);
+ if(p->dhdu) free(p->dhdu);
+ if(p->skyhdu) free(p->skyhdu);
+ if(p->stdhdu) free(p->stdhdu);
+ if(p->stdname) free(p->stdname);
+ if(p->kernelname) free(p->kernelname);
+ if(p->detectionname) free(p->detectionname);
+ if(p->convolvedname) free(p->convolvedname);
if(p->conv!=p->input) gal_data_free(p->conv);
-
- /* Clean up the tile structure. */
- p->ltl.numchannels=NULL;
- gal_tile_full_free_contents(&p->ltl);
- gal_tile_full_free_contents(&p->cp.tl);
+ if(p->clumpsn_s_name) free(p->clumpsn_s_name);
+ if(p->clumpsn_d_name) free(p->clumpsn_d_name);
+ if(p->segmentationname) free(p->segmentationname);
/* Print the final message. */
if(!p->cp.quiet && t1)
diff --git a/bin/noisechisel/ui.h b/bin/segment/ui.h
similarity index 54%
copy from bin/noisechisel/ui.h
copy to bin/segment/ui.h
index 6e33e94..01a67ee 100644
--- a/bin/noisechisel/ui.h
+++ b/bin/segment/ui.h
@@ -1,11 +1,11 @@
/*********************************************************************
-NoiseChisel - Detect and segment signal in a noisy dataset.
-NoiseChisel is part of GNU Astronomy Utilities (Gnuastro) package.
+Segment - Segment initial labels based on signal structure.
+Segment is part of GNU Astronomy Utilities (Gnuastro) package.
Original author:
Mohammad Akhlaghi <address@hidden>
Contributing author(s):
-Copyright (C) 2015-2018, Free Software Foundation, Inc.
+Copyright (C) 2018, Free Software Foundation, Inc.
Gnuastro is free software: you can redistribute it and/or modify it
under the terms of the GNU General Public License as published by the
@@ -40,8 +40,7 @@ along with Gnuastro. If not, see
<http://www.gnu.org/licenses/>.
/* Option groups particular to this program. */
enum program_args_groups
{
- UI_GROUP_DETECTION = GAL_OPTIONS_GROUP_AFTER_COMMON,
- UI_GROUP_SEGMENTATION,
+ UI_GROUP_SEGMENTATION = GAL_OPTIONS_GROUP_AFTER_COMMON,
};
@@ -50,58 +49,39 @@ enum program_args_groups
/* Available letters for short options:
- a b f j l n u x z
- A H J W X Y
+ a b e f g i j l n p r t u w x z
+ A E H J Q R W X Y
*/
enum option_keys_enum
{
/* With short-option version. */
- UI_KEY_LARGETILESIZE = 'L',
UI_KEY_KERNEL = 'k',
- UI_KEY_WIDEKERNEL = 'w',
- UI_KEY_SKYSUBTRACTED = 'E',
+ UI_KEY_DETECTION = 'd',
+ UI_KEY_LARGETILESIZE = 'L',
UI_KEY_MINSKYFRAC = 'B',
- UI_KEY_MIRRORDIST = 'r',
- UI_KEY_MODMEDQDIFF = 'Q',
- UI_KEY_QTHRESH = 't',
- UI_KEY_ERODE = 'e',
- UI_KEY_OPENING = 'p',
- UI_KEY_SIGMACLIP = 's',
- UI_KEY_DTHRESH = 'R',
- UI_KEY_DETSNMINAREA = 'i',
- UI_KEY_DETQUANT = 'c',
- UI_KEY_DETGROWQUANT = 'd',
- UI_KEY_SEGSNMINAREA = 'm',
- UI_KEY_SEGQUANT = 'g',
+ UI_KEY_SNMINAREA = 'm',
+ UI_KEY_SNQUANT = 'c',
UI_KEY_KEEPMAXNEARRIVER = 'v',
+ UI_KEY_CLUMPSNTHRESH = 's',
UI_KEY_GTHRESH = 'G',
UI_KEY_MINRIVERLENGTH = 'y',
UI_KEY_OBJBORDERSN = 'O',
UI_KEY_CONTINUEAFTERCHECK = 'C',
-
/* Only with long version (start with a value 1000, the rest will be set
automatically). */
UI_KEY_KHDU = 1000,
UI_KEY_CONVOLVED,
- UI_KEY_CONVOLVEDHDU,
- UI_KEY_WKHDU,
+ UI_KEY_CHDU,
+ UI_KEY_DHDU,
+ UI_KEY_SKY,
+ UI_KEY_SKYHDU,
+ UI_KEY_STD,
+ UI_KEY_STDHDU,
+ UI_KEY_RAWOUTPUT,
UI_KEY_MINNUMFALSE,
- UI_KEY_ONLYDETECTION,
UI_KEY_GROWNCLUMPS,
- UI_KEY_SMOOTHWIDTH,
- UI_KEY_QTHRESHTILEQUANT,
- UI_KEY_CHECKQTHRESH,
- UI_KEY_ERODENGB,
- UI_KEY_NOERODEQUANT,
- UI_KEY_OPENINGNGB,
- UI_KEY_CHECKDETSKY,
- UI_KEY_CHECKDETSN,
- UI_KEY_DETGROWMAXHOLESIZE,
- UI_KEY_CLEANGROWNDET,
- UI_KEY_CHECKDETECTION,
- UI_KEY_CHECKSKY,
- UI_KEY_CHECKCLUMPSN,
+ UI_KEY_CHECKSN,
UI_KEY_CHECKSEGMENTATION,
};
@@ -110,14 +90,13 @@ enum option_keys_enum
void
-ui_read_check_inputs_setup(int argc, char *argv[],
- struct noisechiselparams *p);
+ui_read_check_inputs_setup(int argc, char *argv[], struct segmentparams *p);
void
-ui_abort_after_check(struct noisechiselparams *p, char *filename,
- char *file2name, char *description);
+ui_abort_after_check(struct segmentparams *p, char *filename, char *file2name,
+ char *description);
void
-ui_free_report(struct noisechiselparams *p, struct timeval *t1);
+ui_free_report(struct segmentparams *p, struct timeval *t1);
#endif
diff --git a/configure.ac b/configure.ac
index 3c61a77..7b589c2 100644
--- a/configure.ac
+++ b/configure.ac
@@ -476,6 +476,12 @@ AC_ARG_ENABLE([noisechisel],
[AS_IF([test "x$enable_noisechisel" != xno],
[enable_noisechisel=yes; ayes=true])],
[enable_noisechisel=notset])
+AC_ARG_ENABLE([segment],
+ [AS_HELP_STRING([--enable-segment],
+ [Install Segment and other enabled programs.])],
+ [AS_IF([test "x$enable_segment" != xno],
+ [enable_segment=yes; ayes=true])],
+ [enable_segment=notset])
AC_ARG_ENABLE([statistics],
[AS_HELP_STRING([--enable-statistics],
[Install Statistics and other enabled programs.])],
@@ -511,41 +517,43 @@ AC_ARG_ENABLE([warp],
# disabled).
AS_IF([test $ayes = true ],
[
- AS_IF([test $enable_arithmetic = notset], [enable_arithmetic=no])
- AS_IF([test $enable_buildprog = notset], [enable_buildprog=no])
- AS_IF([test $enable_convertt = notset], [enable_convertt=no])
- AS_IF([test $enable_convolve = notset], [enable_convolve=no])
- AS_IF([test $enable_cosmiccal = notset], [enable_cosmiccal=no])
- AS_IF([test $enable_crop = notset], [enable_crop=no])
- AS_IF([test $enable_fits = notset], [enable_fits=no])
- AS_IF([test $enable_match = notset], [enable_match=no])
- AS_IF([test $enable_mkcatalog = notset], [enable_mkcatalog=no])
- AS_IF([test $enable_mknoise = notset], [enable_mknoise=no])
- AS_IF([test $enable_mkprof = notset], [enable_mkprof=no])
+ AS_IF([test $enable_arithmetic = notset], [enable_arithmetic=no])
+ AS_IF([test $enable_buildprog = notset], [enable_buildprog=no])
+ AS_IF([test $enable_convertt = notset], [enable_convertt=no])
+ AS_IF([test $enable_convolve = notset], [enable_convolve=no])
+ AS_IF([test $enable_cosmiccal = notset], [enable_cosmiccal=no])
+ AS_IF([test $enable_crop = notset], [enable_crop=no])
+ AS_IF([test $enable_fits = notset], [enable_fits=no])
+ AS_IF([test $enable_match = notset], [enable_match=no])
+ AS_IF([test $enable_mkcatalog = notset], [enable_mkcatalog=no])
+ AS_IF([test $enable_mknoise = notset], [enable_mknoise=no])
+ AS_IF([test $enable_mkprof = notset], [enable_mkprof=no])
AS_IF([test $enable_noisechisel = notset], [enable_noisechisel=no])
- AS_IF([test $enable_statistics = notset], [enable_statistics=no])
- AS_IF([test $enable_table = notset], [enable_table=no])
-# AS_IF([test $enable_TEMPLATE = notset], [enable_TEMPLATE=no])
- AS_IF([test $enable_warp = notset], [enable_warp=no])
+ AS_IF([test $enable_segment = notset], [enable_segment=no])
+ AS_IF([test $enable_statistics = notset], [enable_statistics=no])
+ AS_IF([test $enable_table = notset], [enable_table=no])
+# AS_IF([test $enable_TEMPLATE = notset], [enable_TEMPLATE=no])
+ AS_IF([test $enable_warp = notset], [enable_warp=no])
],
[
- AS_IF([test $enable_arithmetic = notset], [enable_arithmetic=yes])
- AS_IF([test $enable_buildprog = notset], [enable_buildprog=yes])
- AS_IF([test $enable_convertt = notset], [enable_convertt=yes])
- AS_IF([test $enable_convolve = notset], [enable_convolve=yes])
- AS_IF([test $enable_cosmiccal = notset], [enable_cosmiccal=yes])
- AS_IF([test $enable_crop = notset], [enable_crop=yes])
- AS_IF([test $enable_fits = notset], [enable_fits=yes])
- AS_IF([test $enable_match = notset], [enable_match=yes])
- AS_IF([test $enable_mkcatalog = notset], [enable_mkcatalog=yes])
- AS_IF([test $enable_mknoise = notset], [enable_mknoise=yes])
- AS_IF([test $enable_mkprof = notset], [enable_mkprof=yes])
+ AS_IF([test $enable_arithmetic = notset], [enable_arithmetic=yes])
+ AS_IF([test $enable_buildprog = notset], [enable_buildprog=yes])
+ AS_IF([test $enable_convertt = notset], [enable_convertt=yes])
+ AS_IF([test $enable_convolve = notset], [enable_convolve=yes])
+ AS_IF([test $enable_cosmiccal = notset], [enable_cosmiccal=yes])
+ AS_IF([test $enable_crop = notset], [enable_crop=yes])
+ AS_IF([test $enable_fits = notset], [enable_fits=yes])
+ AS_IF([test $enable_match = notset], [enable_match=yes])
+ AS_IF([test $enable_mkcatalog = notset], [enable_mkcatalog=yes])
+ AS_IF([test $enable_mknoise = notset], [enable_mknoise=yes])
+ AS_IF([test $enable_mkprof = notset], [enable_mkprof=yes])
AS_IF([test $enable_noisechisel = notset], [enable_noisechisel=yes])
- AS_IF([test $enable_statistics = notset], [enable_statistics=yes])
- AS_IF([test $enable_table = notset], [enable_table=yes])
-# AS_IF([test $enable_TEMPLATE = notset], [enable_TEMPLATE=yes])
- AS_IF([test $enable_warp = notset], [enable_warp=yes])
+ AS_IF([test $enable_segment = notset], [enable_segment=yes])
+ AS_IF([test $enable_statistics = notset], [enable_statistics=yes])
+ AS_IF([test $enable_table = notset], [enable_table=yes])
+# AS_IF([test $enable_TEMPLATE = notset], [enable_TEMPLATE=yes])
+ AS_IF([test $enable_warp = notset], [enable_warp=yes])
]
)
@@ -562,22 +570,23 @@ AS_IF([test "x$has_gnulibtool" = "xno"],
[enable_buildprog=no])
# Make the enable_package values available for the Makefile:
-AM_CONDITIONAL([COND_ARITHMETIC], [test $enable_arithmetic = yes])
-AM_CONDITIONAL([COND_BUILDPROG], [test $enable_buildprog = yes])
-AM_CONDITIONAL([COND_CONVERTT], [test $enable_convertt = yes])
-AM_CONDITIONAL([COND_CONVOLVE], [test $enable_convolve = yes])
-AM_CONDITIONAL([COND_COSMICCAL], [test $enable_cosmiccal = yes])
-AM_CONDITIONAL([COND_CROP], [test $enable_crop = yes])
-AM_CONDITIONAL([COND_FITS], [test $enable_fits = yes])
-AM_CONDITIONAL([COND_MATCH], [test $enable_match = yes])
-AM_CONDITIONAL([COND_MKCATALOG], [test $enable_mkcatalog = yes])
-AM_CONDITIONAL([COND_MKNOISE], [test $enable_mknoise = yes])
-AM_CONDITIONAL([COND_MKPROF], [test $enable_mkprof = yes])
+AM_CONDITIONAL([COND_ARITHMETIC], [test $enable_arithmetic = yes])
+AM_CONDITIONAL([COND_BUILDPROG], [test $enable_buildprog = yes])
+AM_CONDITIONAL([COND_CONVERTT], [test $enable_convertt = yes])
+AM_CONDITIONAL([COND_CONVOLVE], [test $enable_convolve = yes])
+AM_CONDITIONAL([COND_COSMICCAL], [test $enable_cosmiccal = yes])
+AM_CONDITIONAL([COND_CROP], [test $enable_crop = yes])
+AM_CONDITIONAL([COND_FITS], [test $enable_fits = yes])
+AM_CONDITIONAL([COND_MATCH], [test $enable_match = yes])
+AM_CONDITIONAL([COND_MKCATALOG], [test $enable_mkcatalog = yes])
+AM_CONDITIONAL([COND_MKNOISE], [test $enable_mknoise = yes])
+AM_CONDITIONAL([COND_MKPROF], [test $enable_mkprof = yes])
AM_CONDITIONAL([COND_NOISECHISEL], [test $enable_noisechisel = yes])
-AM_CONDITIONAL([COND_STATISTICS], [test $enable_statistics = yes])
-AM_CONDITIONAL([COND_TABLE], [test $enable_table = yes])
-#AM_CONDITIONAL([COND_TEMPLATE], [test $enable_TEMPLATE = yes])
-AM_CONDITIONAL([COND_WARP], [test $enable_warp = yes])
+AM_CONDITIONAL([COND_SEGMENT], [test $enable_segment = yes])
+AM_CONDITIONAL([COND_STATISTICS], [test $enable_statistics = yes])
+AM_CONDITIONAL([COND_TABLE], [test $enable_table = yes])
+#AM_CONDITIONAL([COND_TEMPLATE], [test $enable_TEMPLATE = yes])
+AM_CONDITIONAL([COND_WARP], [test $enable_warp = yes])
@@ -597,6 +606,7 @@ AC_CONFIG_FILES([Makefile
bin/match/Makefile
bin/mkprof/Makefile
bin/mknoise/Makefile
+ bin/segment/Makefile
bin/convertt/Makefile
bin/convolve/Makefile
bin/buildprog/Makefile
diff --git a/doc/Makefile.am b/doc/Makefile.am
index a18dce2..5b496a5 100644
--- a/doc/Makefile.am
+++ b/doc/Makefile.am
@@ -117,6 +117,9 @@ endif
if COND_NOISECHISEL
MAYBE_NOISECHISEL_MAN = man/astnoisechisel.1
endif
+if COND_SEGMENT
+ MAYBE_SEGMENT_MAN = man/astsegment.1
+endif
if COND_STATISTICS
MAYBE_STATISTICS_MAN = man/aststatistics.1
endif
@@ -129,11 +132,12 @@ endif
#if COND_TEMPLATE
# MAYBE_TEMPLATE_MAN = man/astTEMPLATE.1
#endif
-dist_man_MANS = $(MAYBE_ARITHMETIC_MAN) $(MAYBE_BUILDPROG_MAN) \
- $(MAYBE_CONVERTT_MAN) $(MAYBE_CONVOLVE_MAN) $(MAYBE_COSMICCAL_MAN) \
- $(MAYBE_CROP_MAN) $(MAYBE_FITS_MAN) $(MAYBE_MATCH_MAN) $(MAYBE_WARP_MAN) \
- $(MAYBE_MKCATALOG_MAN) $(MAYBE_MKNOISE_MAN) $(MAYBE_MKPROF_MAN) \
- $(MAYBE_NOISECHISEL_MAN) $(MAYBE_STATISTICS_MAN) $(MAYBE_TABLE_MAN)
+dist_man_MANS = $(MAYBE_ARITHMETIC_MAN) $(MAYBE_BUILDPROG_MAN) \
+ $(MAYBE_CONVERTT_MAN) $(MAYBE_CONVOLVE_MAN) $(MAYBE_COSMICCAL_MAN) \
+ $(MAYBE_CROP_MAN) $(MAYBE_FITS_MAN) $(MAYBE_MATCH_MAN) \
+ $(MAYBE_MKCATALOG_MAN) $(MAYBE_MKNOISE_MAN) $(MAYBE_MKPROF_MAN) \
+ $(MAYBE_NOISECHISEL_MAN) $(MAYBE_SEGMENT_MAN) $(MAYBE_STATISTICS_MAN) \
+ $(MAYBE_TABLE_MAN) $(MAYBE_WARP_MAN)
## See if help2man is present or not. When help2man doesn't exist, we don't
@@ -199,6 +203,10 @@ man/astnoisechisel.1: $(top_srcdir)/bin/noisechisel/args.h
$(ALLMANSDEP)
$(MAYBE_HELP2MAN) -n "detect signal in a noisy image" \
--libtool $(toputildir)/noisechisel/astnoisechisel
+man/astsegment.1: $(top_srcdir)/bin/segment/args.h $(ALLMANSDEP)
+ $(MAYBE_HELP2MAN) -n "segmentation based on signal structure" \
+ --libtool $(toputildir)/segment/astsegment
+
man/aststatistics.1: $(top_srcdir)/bin/statistics/args.h $(ALLMANSDEP)
$(MAYBE_HELP2MAN) -n "calculate statistics of a dataset" \
--libtool $(toputildir)/statistics/aststatistics
diff --git a/doc/gnuastro.texi b/doc/gnuastro.texi
index 001f0ff..9ca3f41 100644
--- a/doc/gnuastro.texi
+++ b/doc/gnuastro.texi
@@ -87,6 +87,9 @@ License''.
* NoiseChisel: (gnuastro)NoiseChisel. Detect signal in noise.
* astnoisechisel: (gnuastro)Invoking astnoisechisel. Options to NoiseChisel.
+* Segment: (gnuastro)Segment. Segment detections based on signal structure.
+* astsegment: (gnuastro)Invoking astsegment. Options to Segment.
+
* Statistics: (gnuastro)Statistics. Get image Statistics.
* aststatistics: (gnuastro)Invoking aststatistics. Options to Statistics.
@@ -425,6 +428,7 @@ Data analysis
* Statistics:: Calculate dataset statistics.
* NoiseChisel:: Detect objects in an image.
+* Segment:: Segment detections based on signal structure.
* MakeCatalog:: Catalog from input and labeled images.
* Match:: Match two datasets.
@@ -450,9 +454,18 @@ Invoking NoiseChisel
* General NoiseChisel options:: General NoiseChisel preprocessing.
* Detection options:: Configure detection in NoiseChisel.
-* Segmentation options:: Configure segmentation in NoiseChisel.
* NoiseChisel output:: NoiseChisel's output format.
+Segment
+
+* Invoking astsegment:: Inputs, outputs and options to Segment
+
+Invoking Segment
+
+* Segment inputs and general settings:: Input files and general options.
+* Segmentation options:: Parameters of the segmentation process.
+* Segment output:: Outputs of Segment
+
MakeCatalog
* Detection and catalog production:: Discussing why/how to treat these
separately
@@ -463,10 +476,10 @@ MakeCatalog
Invoking MakeCatalog
-* MakeCatalog input files:: Specifying the different input files.
-* MakeCatalog general settings:: Options for general column settings.
-* Upper-limit magnitude settings:: Necessary to define upper-limit magnitudes.
-* MakeCatalog output columns:: How to specify the columns in the output.
+* MakeCatalog inputs and basic settings:: Input files and basic settings.
+* Upper-limit settings:: Settings for upper-limit measurements.
+* MakeCatalog output columns:: Available columns in MakeCatalog's output
table.
+* MakeCatalog output file:: File names of MakeCatalog's output table.
Match
@@ -1561,12 +1574,19 @@ separate steps and modularized.
@cindex Announcements
@cindex Mailing list: info-gnuastro
-Gnuastro has a dedicated mailing list for making announcements. Anyone
-that is interested can subscribe to this mailing list to stay up to
-date with new releases or when the dependencies (see
address@hidden) have been updated. To subscribe to this list,
-please visit
address@hidden://lists.gnu.org/mailman/listinfo/info-gnuastro}.
+Gnuastro has a dedicated mailing list for making announcements
+(@code{info-gnuastro}). Anyone can subscribe to this mailing list. Anytime
+there is a new stable or test release, an email will be circulated
+there. The email contains a summary of the overall changes along with a
+detailed list (from the @file{NEWS} file). This mailing list is thus the
+best way to stay up to date with new releases, easily learn about the
+updated/new features, or dependencies (see @ref{Dependencies}).
+
+To subscribe to this list, please visit
address@hidden://lists.gnu.org/mailman/listinfo/info-gnuastro}. Traffic (number
+of mails per unit time) in this list is designed to be very low: only a
+handful of mails per year. Previous annoucements are available on
address@hidden://lists.gnu.org/archive/html/info-gnuastro/, its archive}.
@@ -2479,13 +2499,11 @@ The other sections don't have such shortcuts. To
directly access them from
the command-line, you need to tell Info to look into Gnuastro's manual,
then look for the specific section (an unambiguous title is necessary). For
example, if you only want to review/remember NoiseChisel's @ref{Detection
-options} or @ref{Segmentation options}), just run any of these
-commands. Note how case is irrelevant for Info when calling a title in this
-manner.
+options}), just run the following command. Note how case is irrelevant for
+Info when calling a title in this manner.
@example
$ info gnuastro "Detection options"
-$ info gnuastro "segmentation options"
@end example
In general, Info is a wonderful and powerful way to access this whole book
@@ -4813,13 +4831,15 @@ after you turn on your computer). Therefore you need
administrator or root
privileges to access or modify them.
@item @file{~/.bash_profile}
-If you are using (GNU) Bash as your shell, the commands in this file are run
-once every time you log in to your account.
+If you are using (GNU) Bash as your shell, the commands in this file are
+run, when you log in to your account @emph{through Bash}. Most commonly
+when you login through the virtual console (where there is no graphic user
+interface).
@item @file{~/.bashrc}
If you are using (GNU) Bash as your shell, the commands here will be run
-each time you start a terminal (for example, when you open your terminal
-emulator in the graphic user interface).
+each time you start a terminal and are already logged in. For example, when
+you open your terminal emulator in the graphic user interface.
@end table
@@ -6164,17 +6184,17 @@ configuration doesn't exist in the running version of
the program.
Here is one example of how this option can be used in conjunction with the
@option{--lastconfig} option. Let's assume that you were satisfied with the
-results of this command: @command{astnoisechisel image.fits
---detquant=0.95} (along with various options set in various configuration
-files). You can save the state of NoiseChisel and reproduce that exact
-result on @file{image.fits} later by following these steps (the the extra
-spaces, and @key{\}, are only for easy readability, if you want to try it
-out, only one space between each token is enough).
+results of this command: @command{astnoisechisel image.fits --snquant=0.95}
+(along with various options set in various configuration files). You can
+save the state of NoiseChisel and reproduce that exact result on
address@hidden later by following these steps (the the extra spaces, and
address@hidden, are only for easy readability, if you want to try it out, only
one
+space between each token is enough).
@example
$ echo "onlyversion X.XX" > reproducible.conf
$ echo "lastconfig 1" >> reproducible.conf
-$ astnoisechisel image.fits --detquant=0.95 -P \
+$ astnoisechisel image.fits --snquant=0.95 -P \
>> reproducible.conf
@end example
@@ -12495,6 +12515,7 @@ input dataset (with @ref{MakeCatalog}).
@menu
* Statistics:: Calculate dataset statistics.
* NoiseChisel:: Detect objects in an image.
+* Segment:: Segment detections based on signal structure.
* MakeCatalog:: Catalog from input and labeled images.
* Match:: Match two datasets.
@end menu
@@ -13557,64 +13578,96 @@ one for the Sky standard deviation).
@end table
address@hidden NoiseChisel, MakeCatalog, Statistics, Data analysis
address@hidden NoiseChisel, Segment, Statistics, Data analysis
@section NoiseChisel
@cindex Labeling
@cindex Detection
@cindex Segmentation
-Once instrumental signatures are removed from the raw data in the initial
-reduction process (see @ref{Data manipulation}). We are ready to derive
-scientific results out of them. But we can't do anything special with a raw
-dataset, for example an image is just an array of values. Every pixel just
-has one value and its position within the image. Therefore, the first step
-of your high-level analysis will be to classify/label the dataset
-elements/pixels into two classes: signal and noise. This process is
-formally known as @emph{detection}. Afterwards, you want to separate the
-detections into multiple components (for example when two detected regions
-aren't touching, they should be treated independently as two distant
-galaxies for example). This higher level classification of the detections
-is known as @emph{segmentation}. NoiseChisel is Gnuastro's program for
-detection and segmentation.
-
-NoiseChisel works based on a new noise-based approach to signal detection
-and was introduced to the astronomical community in
address@hidden://arxiv.org/abs/1505.01664, Akhlaghi and Ichikawa
-[2015]}. NoiseChisel's primary output is an array (image) with the same
-size as the input but containing labels: those pixels with a label of 0 are
-noise/sky while those pixels with labels larger than 0 are detections
-(separate segments will be given positive integers, starting from 1). For
-more on NoiseChisel's particular output format and its benefits (especially
-in conjunction with @ref{MakeCatalog}), please see
address@hidden://arxiv.org/abs/1611.06387, Akhlaghi [2016]}. The published
-paper cannot under go any updates, but the NoiseChisel software has
-evolved, you can see the major changes in @ref{NoiseChisel changes after
-publication}.
-
-Data is inherently mixed with noise: only mock/simulated datasets are free
-of noise. So this process of separating signal from noise is not
-trivial. In particular, most scientifically interesting astronomical
-targets are faint, can have a large variety of morphologies along with a
-large distribution in brightness and size which are all drowned in a ocean
-of noise. So detection is a uniquely vital aspect of any scientific work
-and even more so for astronomical research. This is such a fundamental step
-that designing of NoiseChisel was the primary motivation behind creating
-Gnuastro: the first generation of Gnuastro's programs were all first part
-of what later became NoiseChisel, afterwards they spinned-off into separate
-programs.
+Once instrumental signatures are removed from the raw data (image) in the
+initial reduction process (see @ref{Data manipulation}). You are naturally
+eager to start answering the scientific questions that motivated the data
+collection. However, the raw dataset/image is just an array of
+values/pixels, that is all! These raw values cannot directly be used to
+answer your scientific questions (for example ``how many galaxies are there
+in the image?''). Therefore, the first high-level step will be to classify,
+or label, the dataset elements (pixels) into two classes: 1) noise, where
+random effects are the major contributor to the value, and 2) signal, where
+non-random factors (for example light from a distant galaxy) are
+present. This classification of a dataset is formally known as
address@hidden
+
+In a dataset, signal is inherently mixed with noise: only mock/simulated
+datasets are free of noise. Therefore detection, or the process of
+separating signal from noise, determines the number of objects you study
+and the accuracy of any higher-level measurement you do on them. Detection
+is thus the most important step of any analysis and is not trivial. In
+particular, the most scientifically interesting astronomical targets are
+faint, can have a large variety of morphologies, along with a large
+distribution in brightness and size, and are deeply drowned in an ocean of
+noise. Therefore when noise is significant, detection is the uniquely
+decisive step of your scientific result.
@cindex Erosion
-The name of NoiseChisel is derived from the first thing it does after
+NoiseChisel is Gnuastro's program for detection. NoiseChisel uses a
+noise-based paradigm to signal detection that was initially introduced in
address@hidden://arxiv.org/abs/1505.01664, Akhlaghi and Ichikawa [2015]}. The
+name of NoiseChisel is derived from the first thing it does after
thresholding the dataset: to erode it. In mathematical morphology, erosion
on pixels can be pictured as carving off boundary pixels. Hence, what
NoiseChisel does is similar to what a wood chisel or stone chisel do. It is
-just not a hardware, but a software. In fact looking at it as a chisel and
+just not a hardware, but a software. In fact, looking at it as a chisel and
your dataset as a solid cube of rock will greatly help in best using it:
-with NoiseChisel you literally carve the galaxies/stars/comets out of the
-noise. Try running it with the @option{--checkdetection} option to see each
-step of the carving process on your input dataset. You can then change a
-specific option to carve out your signal out of the noise more
-successfully.
+with NoiseChisel you literally carve your targets out of the noise. Try
+running it with the @option{--checkdetection} option to see each step of
+the carving process on your input dataset.
+
address@hidden Segmentation
+NoiseChisel's primary output is a dataset with the same size as the input
+but with integer values that identify each pixel's label (class). Pixels
+that don't harbor any detected signal (noise) are given a label (or value)
+of zero and those with a positive integer have been identified as hosting
+signal. Segmentation is the process of classifing the signal into
+higher-level constructs, for example if you have two separate galaxies in
+one image, you give them separate labels. NoiseChisel does a first order
+segmentation by labeling detections that aren't touching each other as
+separate detections. Each physically separate detection is thus given a
+different label (or counter, starting from 1).
+
+For more on NoiseChisel's particular output format and its benefits
+(especially in conjunction with @ref{MakeCatalog}), please see
address@hidden://arxiv.org/abs/1611.06387, Akhlaghi [2016]}. The output is
+designed to be generic enough to be easily used in any higher-level
+analysis: Higher-level segmentation (for example classifying a large
+detected body as multiple nearby galaxies), you can feed NoiseChisel's
+output to Gnuastro's Segment program (see @ref{Segment}). If NoiseChisel's
+first order segmentation is enough for your analysis, you can feed it
+directly into MakeCatalog to do measurements over the detections and
+produce a catalog (see @ref{MakeCatalog}).
+
+Thanks to the published papers mentioned above, there is no need to provide
+a more complete introduction in this book. However, published papers cannot
+be updated any more, but the software has evolved/changed. The changes
+since publication are documented in @ref{NoiseChisel changes after
+publication}. Afterwards, in @ref{Invoking astnoisechisel}, the details of
+running NoiseChisel and its options are discussed.
+
+As discussed above, detection is the single most important step for your
+scientific result. It is therefore very important to obtain a good
+understanding of NoiseChisel (and afterwards @ref{Segment} and
address@hidden). We thus strongly recommend that after reading the
+papers above and the respective sections of Gnuastro's book, you play a
+little with the settings (in the order presented in @ref{Invoking
+astnoisechisel}) on a dataset you are familiar with and inspect all the
+check images (produced with options starting with @option{--check}) to see
+the effect of each parameter.
+
address@hidden program usage tutorial} is also a good place to get a feeling
+of how Gnuastro's programs are built to complement, and build upon, each
+other. This tutorial culminates culminating in using NoiseChisel to detect
+galaxies and use its outputs to find the galaxy colors. This is a common
+use case, so please also complete that tutorial for most effectively using
+NoiseChisel in conjunction with all of the other Gnuastro programs.
@menu
* NoiseChisel changes after publication:: Changes to the software after
publication.
@@ -13624,42 +13677,52 @@ successfully.
@node NoiseChisel changes after publication, Invoking astnoisechisel,
NoiseChisel, NoiseChisel
@subsection NoiseChisel changes after publication
-Before using NoiseChisel it is strongly recommended to read
address@hidden://arxiv.org/abs/1505.01664, Akhlaghi and Ichikawa [2015]} to
-gain a good understanding of what it does and how each parameter influences
-the output. Thanks to that paper, there is no need to go into the details
-of the major processing steps. Hence we can just dive into the details of
-running NoiseChisel in @ref{Invoking astnoisechisel}.
-
-However, the paper cannot undergo any further updates, but NoiseChisel will
-evolve: better algorithms or steps will be found, thus options will be
-added or removed. So this book is the final and definitive guide. For a
-more detailed list of changes in each release, please follow the
address@hidden file. The @file{NEWS} file is in the released Gnuastro tarball
-(see @ref{Release tarball}). You can also view the most recent @file{NEWS}
-file @url{http://git.savannah.gnu.org/cgit/gnuastro.git/plain/NEWS,
address@hidden online version of the @file{NEWS} file here may
-contain features that have been implemented, but not yet officially
-released as a tarball (see @ref{Downloading the source}). Therefore, please
-be sure to look at the dates and versions above each group of changed
-features and make sure it corresponds to your installed version. It is
-hence recommended to always stay up to date, see @ref{Announcements}).}.
-
-To make the transition form the paper to this book easier (and encourage
-reading the paper), below you can see the major changes since the paper was
-published. First, the options that have been removed are discussed,
-followed by those that have been added.
+NoiseChisel was initially introduced in
address@hidden://arxiv.org/abs/1505.01664, Akhlaghi and Ichikawa [2015]}. It is
+thus strongly recommended to read it for a good understanding of what it
+does and how each parameter influences the output. To help in understanding
+how it works, that paper has a large number of figures showing every step
+on multiple mock and real examples.
+
+However, the paper cannot be updated anymore, but NoiseChisel has evolved
+(and will continue to do so): better algorithms or steps have been found,
+thus options will be added or removed. This book is thus the final and
+definitive guide to NoiseChisel. For a more detailed list of changes in
+each release, please follow the @file{NEWS} file. The @file{NEWS} file is
+present in the released Gnuastro tarball (see @ref{Release tarball}). It is
+also included as a post-script to Gnuastro's announcements, see
address@hidden
+
+The most important change since the publication of that papaer is that
+NoiseChisel is now focused on detection: in spirit with Gnuastro's modular
+design (see @ref{Program design philosophy}), segmentation of the detected
+signal has been spinned-off to Gnuastro's Segment program (see
address@hidden). Below you can see the major changes since that paper was
+published. First, the removed options/features are discussed, then we
+review the new features that have been added.
@noindent
-Removed options:
+Removed features/options:
@itemize
+
@item
address@hidden: This option was used to account for the extra
+noise that is added if the Sky value has already been subtracted. However,
+NoiseChisel estimates the Sky standard deviation based on the input data,
+not exposure maps or other theoretical considerations. Therefore the
+standard deviation of the undetected pixels also contains the errors due to
+any previous sky subtraction. This option is therefore no longer present in
+NoiseChisel.
+
@option{--dilate}: In the paper, true detections were dilated for a final
-dig into the noise. However, simple 8-connected dilation can produce boxy
-results which are not realistic and could miss diffuse flux. The final dig
+dig into the noise. However, the simple 8-connected dilation produced boxy
+results which were not realistic and could miss diffuse flux. The final dig
into the noise is now done by ``grow''ing the true detections, similar to
how true clumps were grown, see the description of @option{--detgrowquant}
below and in @ref{Detection options} for more on the new alternative.
+
address@hidden
+Segmentation has been completely moved to a new program: @ref{Segment}.
@end itemize
@noindent
@@ -13673,8 +13736,9 @@ thresholds (see @ref{Quantifying signal in a tile}).
Until now, NoiseChisel
would convolve an image once and estimate the proper tiles for quantile
estimations on the convolved image. The same convolved image would later be
used for quantile estimation. A larger kernel does increase the skewness
-(and thus difference between the mode and median), however, it disfigures
-the shapes/morphology of the objects.
+(and thus difference between the mode and median, therefore helps in
+detecting the presence signal), however, it disfigures the
+shapes/morphology of the objects.
This new @option{--widekernel} option (and a corresponding @option{--wkhdu}
option to specify its HDU) option are added to solve such cases. When its
@@ -13724,10 +13788,11 @@ of false detections, see the descriptions under this
option in
@node Invoking astnoisechisel, , NoiseChisel changes after publication,
NoiseChisel
@subsection Invoking NoiseChisel
-NoiseChisel will detect and segment signal in noise producing a
-multi-extension labeled image, ready for input into @ref{MakeCatalog} to
-generate a catalog or other processing. The executable name is
address@hidden with the following general template
+NoiseChisel will detect signal in noise producing a multi-extension labeled
+image. Its output can be readily used for input into @ref{Segment}, for
+higher-level segmentation, or @ref{MakeCatalog} to generate a catalog. The
+executable name is @file{astnoisechisel} with the following general
+template
@example
$ astnoisechisel [OPTION ...] InputImage.fits
@@ -13737,9 +13802,12 @@ $ astnoisechisel [OPTION ...] InputImage.fits
One line examples:
@example
-## Detect signal in input.fits:
+## Detect signal in input.fits.
$ astnoisechisel input.fits
+## Inspect all the detection steps after changing a parameter.
+$ astnoisechisel input.fits --qthresh=0.4 --checkdetection
+
## Detect signal assuming input has 4 channels along first dimension
## and 1 along the second. Also set the regular tile size to 100 along
## both dimensions:
@@ -13755,10 +13823,16 @@ shares a large set of common operations with other
Gnuastro programs,
mainly regarding input/output, general processing steps, and general
operating modes. To help in a unified experience between all of Gnuastro's
programs, these operations have the same command-line options, see
address@hidden options} for a full list. Since the common options are
-thoroughly discussed there, they are no longer reviewed here. You can see
-all the options with a short description on the command-line with the
address@hidden option, see @ref{Getting help}.
address@hidden options} for a full list/description (they are not repeated
+here).
+
+As in all Gnuastro programs, options can also be given to NoiseChisel in
+configuration files. For a thorough description on Gnuastro's configuration
+file parsing, please see @ref{Configuration files}. All of NoiseChisel's
+options with a short description are also always available on the
+command-line with the @option{--help} option, see @ref{Getting help}. To
+inspect the option values without actually running NoiseChisel, append your
+command with @option{--printparams} (or @option{-P}).
NoiseChisel's input image may contain blank elements (see @ref{Blank
pixels}). Blank elements will be ignored in all steps of NoiseChisel. Hence
@@ -13776,18 +13850,22 @@ be used. The default kernel is a 2D Gaussian with a
FWHM of 2 pixels
truncated at 5 times the FWHM. This choice of the default kernel is
discussed in Section 3.1.1 of @url{https://arxiv.org/abs/1505.01664,
Akhlaghi and Ichikawa [2015]}. See @ref{Convolution kernel} for kernel
-related options.
+related options. Passing @code{none} to @option{--kernel} will disable
+convolution. On the other hand, through the @option{--convolved} option,
+you may provide an already convolved image, see descriptions below for
+more.
NoiseChisel defines two tessellations over the input (see
@ref{Tessellation}). This enables it to deal with possible gradients in the
input dataset and also significantly improve speed by processing each tile
-on different threads. The tessellation related options are discussed in
address@hidden options}. In particular, NoiseChisel uses two tessellations
-(with everything between them identical except the tile sizes): a
-fine-grained one with smaller tiles (mainly used in detection) and a more
-larger tiled one which is used for multi-threaded processing. The common
+on different threads simultaneously. Tessellation related options are
+discussed in @ref{Processing options}. In particular, NoiseChisel uses two
+tessellations (with everything between them identical except the tile
+sizes): a fine-grained one with smaller tiles (used in thresholding and Sky
+value estimations) and another with larger tiles which is used for
+pseudo-detections over non-detected regions of the image. The common
Tessellation options described in @ref{Processing options} define all
-parameters of both tessellations, only the large tile size for the latter
+parameters of both tessellations. The large tile size for the latter
tessellation is set through the @option{--largetilesize} option. To inspect
the tessellations on your input dataset, run NoiseChisel with
@option{--checktiles}.
@@ -13795,41 +13873,43 @@ the tessellations on your input dataset, run
NoiseChisel with
@cartouche
@noindent
@strong{Usage TIP:} Frequently use the options starting with
address@hidden Depending on what you want to detect in the data, you can
-often play with the parameters/options for a better result than the default
-parameters. You can start with @option{--checkdetection} and
address@hidden for the main steps. For their full list please
-run:
address@hidden Since the noise properties differ between different
+datasets, you can often play with the parameters/options for a better
+result than the default parameters. You can start with
address@hidden for the main steps. For the full list of
+NoiseChisel's checking options please run:
@example
$ astnoisechisel --help | grep check
@end example
@end cartouche
-In the sections below, NoiseChisel's options are classified into three
-general classes to help in easy navigation. @ref{General NoiseChisel
-options} mainly discusses the options relating to input and those that are
-shared in both detection and segmentation. Options to configure the
-detection are described in @ref{Detection options} and @ref{Segmentation
-options} we discuss how you can fine-tune the segmentation of the
-detections. Finally in @ref{NoiseChisel output} the format of NoiseChisel's
-output is discussed. The order of options here follow the same logical
-order that the respective action takes place within NoiseChisel (note that
-the output of @option{--help} is sorted alphabetically).
+Below, we'll discuss NoiseChisel's options, classified into two general
+classes, to help in easy navigation. @ref{General NoiseChisel options}
+mainly discusses the options relating to inputs and prior to
+detection. Afterwards, @ref{Detection options} fully describes every
+configuration parameter (option) related to detection and how they affect
+the final result. The order of options in this section follow the logical
+order within NoiseChisel. On first reading (while you are still new to
+NoiseChisel), it is therefore strongly recommended to read the options in
+the given order below. The output of @option{--printparams} (or
address@hidden) also has this order. However, the output of @option{--help} is
+sorted alphabetically. Finally, in @ref{NoiseChisel output} the format of
+NoiseChisel's output is discussed.
@menu
* General NoiseChisel options:: General NoiseChisel preprocessing.
* Detection options:: Configure detection in NoiseChisel.
-* Segmentation options:: Configure segmentation in NoiseChisel.
* NoiseChisel output:: NoiseChisel's output format.
@end menu
@node General NoiseChisel options, Detection options, Invoking astnoisechisel,
Invoking astnoisechisel
@subsubsection General NoiseChisel options
-The options discussed in this section are mainly regarding the input(s),
-output, and some general processing options that are shared between both
-detection and segmentation. Recall that you can always see the full list of
-Gnuastro's options with the @option{--help} option.
+The options here can be used to configure the inputs and output of
+NoiseChisel, along with some general processing options. Recall that you
+can always see the full list of Gnuastro's options with the @option{--help}
+(see @ref{Getting help}), or @option{--printparams} (or @option{-P}) to see
+their values (see @ref{Operating mode options}).
@table @option
@@ -13840,35 +13920,46 @@ File name of kernel to smooth the image before
applying the threshold, see
value of @option{none}.
The first step of NoiseChisel is to convolve/smooth the image and use the
-convolved image in multiple steps during the processing. It will be used to
-define (and later apply) the quantile threshold (see
address@hidden). The convolved image is also used to define the clumps
-(see Section 3.2.1 and Figure 8 of @url{https://arxiv.org/abs/1505.01664,
-Akhlaghi and Ichikawa [2015]}).
+convolved image in multiple steps including the finding and applying of the
+quantile threshold (see @option{--qthresh}).
+
+The @option{--kernel} option is not mandatory. If not called, a 2D Gaussian
+profile with a FWHM of 2 pixels truncated at 5 times the FWHM is used. This
+choice of the default kernel is discussed in Section 3.1.1 of Akhlaghi and
+Ichikawa [2015]. You can use MakeProfiles to build a kernel with any of its
+recognized profile types and parameters. For more details, please see
address@hidden output dataset}. For example, the command below will make
+a Moffat kernel (with @mymath{\beta=2.8}) with FWHM of 2 pixels truncated
+at 10 times the FWHM.
-The @option{--kernel} option is not mandatory. If no kernel is provided, a
-2D Gaussian profile with a FWHM of 2 pixels truncated at 5 times the FWHM
-is used. This choice of the default kernel is discussed in Section 3.1.1 of
-Akhlaghi and Ichikawa [2015].
address@hidden
+$ astmkprof --oversample=1 --kernel=moffat,2,2.8,10
address@hidden example
+
+Since convolution can be the slowest step of NoiseChisel, for large
+datasets, you can convolve the image once with Gnuastro's Convolve (see
address@hidden), and use the @option{--convolved} option to feed it
+directly to NoiseChisel. This can help getting faster results when you are
+playing/testing the higher-level options.
@item --khdu=STR
HDU containing the kernel in the file given to the @option{--kernel}
option.
@item --convolved=STR
-Use this file as the convolved image and don't do convolution and ignore
address@hidden NoiseChisel will just check the size of the given
+Use this file as the convolved image and don't do convolution (ignore
address@hidden). NoiseChisel will just check the size of the given
dataset is the same as the input's size. If a wrong image (with the same
size) is given to this option, the results (errors, bugs, and etc) are
un-predictable. So please use this option with care and in a highly
-controlled environment. On such scenario is discussed below.
-
-In almost all situations, as the input gets larger, the single most CPU and
-time consuming step in NoiseChisel is convolution (the first step in its
-processing). The process of testing NoiseChisel for the best parameters in
-a given analysis will involve running NoiseChisel multiple times: to see
-the effect of each new option value. Therefore, once the kernel is
-finalized, re-convolving the input will greatly hinder fast testing of
+controlled environment, for example in the scenario discussed below.
+
+In almost all situations, as the input gets larger, the single most CPU
+(and time) consuming step in NoiseChisel is convolution (the first step in
+its processing). To test NoiseChisel for the best parameters in a given
+analysis, you have to run NoiseChisel multiple times and see the effect of
+each change. Therefore, once the kernel is finalized, re-convolving the
+input on every change will greatly hinder, or discourage, testing of
higher-level parameters. With this option, you can convolve the input image
with your chosen kernel once before running NoiseChisel, then feed it to
NoiseChisel on each test run and thus save valuable time for better/more
@@ -13883,8 +13974,9 @@ will appear on the edges, see @ref{Spatial vs.
Frequency domain}.
Below you can see an example of such a scenario: you want to see how
variation of the growth level (through the @option{--detgrowquant} option)
will affect the final result. Recall that you can ignore all the extra
-spaces, new lines, and address@hidden' if you are typing in the terminal (in a
-shell script, remove the @code{$} signs at the start of the lines).
+spaces, new lines, and backslash's (address@hidden') if you are typing in the
+terminal (in a shell script, remove the @code{$} signs at the start of the
+lines).
@example
## Make the kernel to convolve with.
@@ -13903,7 +13995,7 @@ $ for g in 60 65 70 75 80 85 90; do
\
address@hidden --convolvedhdu=STR
address@hidden --chdu=STR
The HDU/extension containing the convolved image in the file given to
@option{--convolved}.
@@ -13913,73 +14005,17 @@ File name of a wider kernel to use in estimating the
difference of the mode
and median in a tile (this difference is used to identify the significance
of signal in that tile, see @ref{Quantifying signal in a tile}). This
convolved image is only used for this purpose. Once the mode is found to be
-sufficiently close to the median, the input convolved with the sharper
-kernel (@option{--kernel}) is used to identify the quantile threshold (see
+sufficiently close to the median, the quantile threshold is found on the
+image convolved with the sharper kernel (@option{--kernel}), see
@option{--qthresh}).
-Since this convolution will significantly slow down the processing, it is
-optional. If no image is given, the mode and median will be found on the
-image that is convolved with the dataset in @option{--kernel}.
+Since convolution will significantly slow down the processing, this option
+is optional. If it isn't given, a single convolved image will be used in
+all situations.
address@hidden --wkhdu=STR
address@hidden --whdu=STR
HDU containing the kernel file given to the @option{--widekernel} option.
address@hidden -E
address@hidden --skysubtracted
-If this option is called, it is assumed that the image has already been sky
-subtracted once. Knowing if the sky has already been subtracted once or not
-is very important in estimating the Signal to noise ratio of the detections
-and clumps. In short an extra @mymath{\sigma_{sky}^2} must be added in the
-error (noise or denominator in the Signal to noise ratio) for every flux
-value that is present in the calculation. This can be interpreted as the
-error in measuring that sky value when it was subtracted by any other
-program. See Section 3.3 in @url{https://arxiv.org/abs/1505.01664, Akhlaghi
-and Ichikawa [2015]}) for a complete explanation.
-
address@hidden -B FLT
address@hidden --minskyfrac=FLT
-Minimum fraction (value between 0 and 1) of sky (undetected) areas in a
-tile for it to be considered in measuring the following detection and
-segmentation properties.
-
address@hidden
-
address@hidden
-Measuring the Signal to noise ratio of false detections during the false
-detection removal on small tiles.
-
address@hidden
-Measuring the sky value (average of undetected pixels) on small tiles. Both
-before the removal of false detections and after it.
-
address@hidden
-Clump Signal to noise ratio in the sky regions of large files.
-
address@hidden itemize
-
-Because of the PSF and their intrinsic amorphous properties, astronomical
-objects (except cosmic rays) never have a clear cutoff and commonly sink
-into the noise very slowly. Even below the very low thresholds used by
-NoiseChisel. So when a large fraction of the area of one mesh is covered by
-detections, it is very plausible that their faint wings are present in the
-undetected regions (hence causing a bias in any measurement). To get an
-accurate measurement of the above parameters over the tessellation, tiles
-that harbor too many detected regions should be excluded. The used tiles
-are visible in the respective @option{--check} option of the given step.
-
address@hidden --minnumfalse=INT
-The minimum number of `pseudo-detections' (to identify false initial
-detections) or clumps (to identifying false clumps) found over the
-un-detected regions to identify a Signal-to-Noise ratio threshold.
-
-The Signal to noise ratio (S/N) of false pseudo-detections and clumps in
-each tile is found using the quantile of the S/N distribution of the
-psudo-detections and clumps over the undetected pixels in each mesh. If the
-number of S/N measurements is not large enough, the quantile will not be
-accurate (can have large scatter). For example if you set
address@hidden (or the top 1 percent), then it is best to have at
-least 100 S/N measurements.
-
@item -L INT[,INT]
@itemx --largetilesize=INT[,INT]
The size of each tile for the tessellation with the larger tile
@@ -13988,53 +14024,63 @@ tessellation are taken from the common options
described in @ref{Processing
options}. The format is identical to that of the @option{--tilesize} option
that is discussed in that section.
address@hidden --onlydetection
-If this option is called, no segmentation will be done and the output will
-only have four extensions (no clumps extension, see @ref{NoiseChisel
-output}). The second extension of the output is not going to be objects but
-raw detections (a large region will be given one label): labeling is only
-done based on connectivity. The last two extensions of the output will be
-the Sky and its Standard deviation.
-
-This option can result in faster processing when only the noise properties
-of the image are desired for a catalog using another image's labels for
-example. A common case is when you want to measure colors or SEDs in
-several images. Let's say you have images in two colors: A and B. For
-simplicity also assume that they are exactly on the same position in the
-sky with the same pixel scale.
-
-You choose to set A as a reference, so you run the NoiseChisel fully on
-A. Then you run NoiseChisel on B with @option{--onlydetection} since you
-only need the noise properties of B (for the signal to noise column in its
-catalog). You can then run MakeCatalog on A normally, see
address@hidden To run MakeCatalog on B, you simply set the object and
-clump labels images to those that NoiseChisel produced for A, see
address@hidden astmkcatalog}.
-
@item --continueaftercheck
Continue NoiseChisel after any of the options starting with
@option{--check}. NoiseChisel involves many steps and as a result, there
are many checks, allowing to inspect the status of the processing. The
-results of each step affect the next steps of processing, so, when you are
-want to check the status of the processing at one step, the time spent to
-complete NoiseChisel is just wasted/distracting time.
+results of each step affect the next steps of processing. Therefore, when
+you want to check the status of the processing at one step, the time spent
+to complete NoiseChisel is just wasted/distracting time.
To encourage easier experimentation with the option values, when you use
any of the NoiseChisel options that start with @option{--check},
-NoiseChisel will abort once all the desired check file(s) is (are)
-completed. If you call the @option{--continueaftercheck} option, you can
-disable this behavior and ask NoiseChisel to continue with the rest of the
-processing after completing the check file(s).
+NoiseChisel will abort once all the desired check file is complete. With
+the @option{--continueaftercheck} option, you can disable this behavior and
+ask NoiseChisel to continue with the rest of the processing, even after
+completing the requested check files are complete.
+
address@hidden --rawoutput
+Don't include the Sky-subtracted input image as the first extension of the
+output. By default, the Sky-subtracted input image is put in the first
+extension of the output. The next extensions are NoiseChisel's main output:
+a labeled image with the detected pixels having positive values and the Sky
+and Sky standard deviation images, see @ref{NoiseChisel output}.
+
+The extra Sky-subtracted input can help in checking NoiseChisel's
+configuration by comparing the detections with the input image: visually
+see if everything you expected is detected (reasonable completeness) and
+that you don't have too many false detections (reasonable purity). This
+visual inspection is simplified if you use SAO DS9 to view NoiseChisel's
+output as a multi-extension datacube, see @ref{Viewing multiextension FITS
+images}.
+
+However, when you are satisfied with your NoiseChisel configuration
+(therefore you don't need to check on every run), the datasets become
+large, or you are running NoiseChisel as part of a pipeline, this
+Sky-subtracted input image can be significant burden (take up a large
+volume). In such cases, @option{--rawoutput} can be used to avoid this
+extra extension. It is always possible to easily produce the Sky-subtracted
+image from the input (assuming it is in extension @code{1} of
address@hidden) and the @code{SKY} extension of NoiseChisel's output (let's
+call it @file{nc.fits}) with a command like below (assuming NoiseChisel
+wasn't run with @option{--oneelempertile}, see @ref{Tessellation}):
+
address@hidden
+$ astarithmetic in.fits nc.fits - -h1 -hSKY
address@hidden example
+
@end table
address@hidden Detection options, Segmentation options, General NoiseChisel
options, Invoking astnoisechisel
address@hidden Detection options, NoiseChisel output, General NoiseChisel
options, Invoking astnoisechisel
@subsubsection Detection options
Detection is the process of separating the pixels in the image into two
groups: 1) Signal and 2) Noise. Through the parameters below, you can
customize the detection process in NoiseChisel. Recall that you can always
-see the full list of Gnuastro's options with the @option{--help} option.
+see the full list of Gnuastro's options with the @option{--help} (see
address@hidden help}), or @option{--printparams} (or @option{-P}) to see
+their values (see @ref{Operating mode options}).
@table @option
@@ -14200,6 +14246,24 @@ it is interpreted as tolerance and if it is larger
than one it is assumed
to be the fixed number of iterations. Hence, in the latter case the value
must be an integer.
address@hidden -B FLT
address@hidden --minskyfrac=FLT
+Minimum fraction (value between 0 and 1) of Sky (undetected) areas in a
+tile. Only tiles with a fraction of undetected pixels (Sky) larger than
+this value will be used to estimate the Sky value. NoiseChisel uses this
+option value twice to estimate the Sky value: after initial detections and
+in the end when false detections have been removed.
+
+Because of the PSF and their intrinsic amorphous properties, astronomical
+objects (except cosmic rays) never have a clear cutoff and commonly sink
+into the noise very slowly. Even below the very low thresholds used by
+NoiseChisel. So when a large fraction of the area of one mesh is covered by
+detections, it is very plausible that their faint wings are present in the
+undetected regions (hence causing a bias in any measurement). To get an
+accurate measurement of the above parameters over the tessellation, tiles
+that harbor too many detected regions should be excluded. The used tiles
+are visible in the respective @option{--check} option of the given step.
+
@item --checkdetsky
Check the initial approximation of the sky value and its standard deviation
in a FITS file ending with @file{_detsky.fits}. With this option,
@@ -14223,8 +14287,8 @@ ratio of the resulting `psudo-detections' are used to
identify true
vs. false detections. See Section 3.1.5 and Figure 7 in Akhlaghi and
Ichikawa (2015) for a very complete explanation.
address@hidden -i INT
address@hidden --detsnminarea=INT
address@hidden -m INT
address@hidden --snminarea=INT
The minimum area to calculate the Signal to noise ratio on the
psudo-detections of both the initially detected and undetected
regions. When the area in a psudo-detection is too small, the Signal to
@@ -14233,7 +14297,7 @@ be heavily skewed to the positive. So it is best to
ignore any
psudo-detection that is smaller than this area. Use
@option{--detsnhistnbins} to check if this value is reasonable or not.
address@hidden --checkdetsn
address@hidden --checksn
Save the S/N values of the pseudo-detections and dilated detections into
three files ending with @file{_detsn_sky.XXX}, @file{_detsn_det.XXX}, and
@file{_detsn_dilated.XXX}. The @file{.XXX} is determined from the
@@ -14250,8 +14314,18 @@ created. This allows you to inspect the steps leading
to the final quantile
threshold, this behavior (to abort NoiseChisel) can be disabled with
@option{--continueaftercheck}.
address@hidden --minnumfalse=INT
+The minimum number of `pseudo-detections' over the un-detected regions to
+identify a Signal-to-Noise ratio threshold. The Signal to noise ratio (S/N)
+of false pseudo-detections in each tile is found using the quantile of the
+S/N distribution of the psudo-detections over the undetected pixels in each
+mesh. If the number of S/N measurements is not large enough, the quantile
+will not be accurate (can have large scatter). For example if you set
address@hidden (or the top 1 percent), then it is best to have at
+least 100 S/N measurements.
+
@item -c FLT
address@hidden --detquant=FLT
address@hidden --snquant=FLT
The quantile of the Signal to noise ratio distribution of the
psudo-detections in each mesh to use for filling the large mesh grid. Note
that this is only calculated for the large mesh grids that satisfy the
@@ -14301,8 +14375,8 @@ the derived pseudo-detection S/N limit, that detection
will be
discarded. In an ideal/clean noise, a true detection's S/N should be larger
than its constituent pseudo-detections because its area is larger and it
also covers more signal. However, on a false detections (especially at
-lower @option{--detquant} values), the increase in size can cause a
-decrease in S/N below that threshold.
+lower @option{--snquant} values), the increase in size can cause a decrease
+in S/N below that threshold.
This will improve purity and not change completeness (a true detection will
not be discarded). Because a true detection has flux in its vicinity and
@@ -14338,39 +14412,406 @@ have the same pixel size as the input, but with the
@end table
address@hidden Segmentation options, NoiseChisel output, Detection options,
Invoking astnoisechisel
+
+
+
address@hidden NoiseChisel output, , Detection options, Invoking astnoisechisel
address@hidden NoiseChisel output
+
+The main output of NoiseChisel is the detected pixels (labeled with
+integers starting from 1) over the input dataset along with the Sky and Sky
+standard deviation that are all returned as multiple extensions of a FITS
+file. The name of the output file can be set by giving a value to
address@hidden If @option{--output} is not used, the output will be
+input name but with a @file{_labeled.fits} suffix, see @ref{Automatic
+output}. If any of the options starting with @option{--check*} are given,
+NoiseChisel won't complete and will abort as soon as the respective check
+images are created. See the descriptions in @ref{Detection options} for a
+description of the extensions in each of the check images when the
address@hidden options are called. With the
address@hidden option, NoiseChisel will produce the check
+images without aborting and will continue to finish the detection.
+
+The default output can have three or four extensions based on the
address@hidden option. If it is called on the command-line or given a
+value of 1 in any of the configuration files, then the output will only
+contain the three raw outputs (Labeled image, Sky and Sky Standard
+deviation). This can be useful when you are calling NoiseChisel as part of
+a large script and on large files where the extra extension is redundant,
+but will take up a large volume.
+
+When @option{--rawoutput} is not called, the first extension will be the
+input image subtracted by the Sky value. When playing-with/testing
+NoiseChisel on a new dataset, having this extra image can greatly simplify
+the inspection of the results. By flipping through the extensions, you can
+check how accurate or effective the change in parameters was.
+
+To inspect NoiseChisel's output, you can configure SAO DS9 in your Graphic
+User Interface (GUI) to open NoiseChisel's output as a multi-extension data
+cube. This will allow you to flip through the different extensions and
+visually inspect the results. This process has been described for the GNOME
+GUI (most common GUI in GNU/Linux operating systems) in @ref{Viewing
+multiextension FITS images}.
+
+The main output (present in any case) is thus the labeled dataset. In this
+image, the non-detected (Sky) pixels of the input are given a label of zero
+and the detected pixel are given positive label (starting from one, based
+on connectivity). The total number of labels is stored as the value to the
address@hidden keyword in the header of this extension. This number (along
+with other important information) is also printed by NoiseChisel in verbose
+mode (when @option{--quiet} is not called).
+
+The last two extensions of the output are the Sky and its Standard
+deviation, see @ref{Sky value} for a complete explanation. They are
+calculated on the tile grid that you defined for NoiseChisel. By default
+these datasets will have the same size as the input, but with all the
+pixels in one tile given one value. To be more space-efficient (keep only
+one pixel per tile), you can use the @option{--oneelempertile} option, see
address@hidden
+
address@hidden
address@hidden
address@hidden Compression
address@hidden space:} with the @option{--rawoutput} and
address@hidden, NoiseChisel's output will only be one labeled
+datasets (only containing integers) and two much smaller arrays with one
+value per tile. Since they have no noise, integer datasets can be
+compressed very effectively (without any loss of data) with exceptionally
+high compression ratios. You can therefore use the following command to
+keep NoiseChisel's output in a much more efficient manner (take up less
+volume).
+
address@hidden GNU Gzip
address@hidden
+$ gzip --best noisechisel_output.fits
address@hidden example
+
address@hidden
+The resulting @file{.fits.gz} file can then be fed into any of Gnuastro's
+programs directly, without having to decompress it separately (it will just
+take them a little longer, because they have to decompress it before use).
+
+Alternatively, you can also tell NoiseChisel to directly build a
address@hidden output (for example with @option{--output=nc.fits.gz}). But
+CFITSIO (which reads and writes FITS files for Gnuastro) puts more emphasis
+on compression/decompression speed. So it doesn't use the most efficient
+compression settings (which can take slightly longer).
address@hidden cartouche
+
+
+
+
+
+
+
+
+
+
address@hidden Segment, MakeCatalog, NoiseChisel, Data analysis
address@hidden Segment
+
+Once signal is separated from noise (for example with @ref{NoiseChisel}),
+you have a binary dataset: each pixel is either signal (1) or noise
+(0). Signal (for example every galaxy in your image) has been ``detected'',
+but they all have a label of 1. This still doesn't answer our scientific
+questions, for example, to count how many galaxies there are in the image
+and measure their properties. At the lowest level, detection is a kind of
+segmentation (segmenting the the whole dataset into signal and noise, see
address@hidden). Here, we'll define segmentation only on signal: to
+separate and find sub-structure within the detections.
+
address@hidden Connected component labeling
+If the targets are clearly separated in the dataset (image), a simple
address@hidden
address@hidden@url{https://en.wikipedia.org/wiki/Connected-component_labeling}}
+algorithm (very basic segmentation) is enough to separate the regions that
+are touching/connected. This is such a basic form of segmentation that
+detection programs (for example NoiseChisel) do this for you. However, in
+most real situations our targets are not nicely separated: they touch (for
+example merging galaxies), or are simply in the same line-of-sight, causing
+their images to overlap. In particular, when you do your detection with
+NoiseChisel, you will detect signal to very low surface brightness limits
+(the faint wings of galaxies or bright stars). Therefore, it often happens
+that several galaxies are detected as one large detection. To continue your
+scientific analysis, it is therefore necessary to separate/segment each
+detection into multiple targets as best as possible.
+
+Segment will use a detection map and its corresponding dataset to find
+sub-structure over the detected areas. Segment was originally part of
address@hidden To help in modularity and improved creativity,
+NoiseChisel's segmentation features have been brought into a separate
+program. Therefore, as with NoiseChisel, the best place to start reading
+about Segment and understanding what it does (with many illustrative
+figures) is Section 3.2 of @url{https://arxiv.org/abs/1505.01664, Akhlaghi
+and Ichikawa [2015]}.
+
+Segment will first find the @emph{true} local maxima over the detections in
+the given dataset along with all the pixels in their vicinity before
+reaching a local minimum. These regions over a detection are called
address@hidden Afterwards, the clumps are grown up to a certain
+threshold. Based on the connectivity of these grown clumps, they are
+considered parts of one @emph{object} or as separate @emph{object}s. See
+Section 3.2 of Akhlaghi and Ichikawa [2015] (link above) for more. In
+short, Segment's main output are two labeled datasets: 1) clumps, and 2)
+objects. The the clumps output will only have positive integers on the true
+clumps, the rest of the detected regions will get a fixed negative
+value. The clump labels start counting from 1 within their host object, so
+if two objects have two and four clumps respectively, the clumps in the
+first will will be labeled 1 and 2 and in the second, they will be labeled
+1, 2, 3, and 4. The objects output has a positive integer (counting from 1)
+for all detected pixels.
+
+
address@hidden
+* Invoking astsegment:: Inputs, outputs and options to Segment
address@hidden menu
+
address@hidden Invoking astsegment, , Segment, Segment
address@hidden Invoking Segment
+
+Segment will identify substructure within a detected region using its
+corresponding image. Its output can be readily used for input into
address@hidden to generate a catalog of the objects and clumps. The
+executable name is @file{astsegment} with the following general
+template
+
address@hidden
+$ astsegment [OPTION ...] InputImage.fits
address@hidden example
+
address@hidden
+One line examples:
+
address@hidden
+## Segment NoiseChisel's detected regions.
+$ astsegment full-noisechisel-output.fits
+
+## Use a hand-input S/N value for keeping true clumps
+## (avoid finding the S/N using the undetected regions).
+$ astsegment nc-out.fits --clumpsnthresh=10
+
+## Inspect all the segmentation steps after changing a parameter.
+$ astsegment input.fits --snquant=0.9 --checksegmentaion
address@hidden example
+
address@hidden Gaussian
address@hidden
+If Segment is to do processing (for example you don't want to get help, or
+see the values to each input parameter), an input image should be provided
+with the recognized extensions (see @ref{Arguments}). NoiseChisel shares a
+large set of common operations with other Gnuastro programs, mainly
+regarding input/output, general processing steps, and general operating
+modes. To help in a unified experience between all of Gnuastro's programs,
+these operations have the same command-line options, see @ref{Common
+options} for a full list/description (they are not repeated here).
+
+As in all Gnuastro programs, options can also be given to Segment in
+configuration files. For a thorough description on Gnuastro's configuration
+file parsing, please see @ref{Configuration files}. All of Segment's
+options with a short description are also always available on the
+command-line with the @option{--help} option, see @ref{Getting help}. To
+inspect the option values without actually running Segment, append your
+command with @option{--printparams} (or @option{-P}).
+
+To help in easy navigation between Segment's options, they are separated
+discussed in two separate sub-sections: @ref{Segment inputs and general
+settings} discusses how you can customize the inputs to Segment along with
+other general Segment options. Another section (@ref{Segmentation options})
+is devoted to options specific to the segmentation process. Finally, in
address@hidden output}, we'll discuss the format of Segment's output file.
+
address@hidden
+* Segment inputs and general settings:: Input files and general options.
+* Segmentation options:: Parameters of the segmentation process.
+* Segment output:: Outputs of Segment
address@hidden menu
+
address@hidden Segment inputs and general settings, Segmentation options,
Invoking astsegment, Invoking astsegment
address@hidden Segment inputs and general settings
+
+Segment needs atleast three datasets as input: 1) Sky-subtracted values
+(actual image), 2) detection labels, and 3) standard deviation. If the
+values dataset is not Sky-subtracted, you can ask Segment to subtract the
+Sky from it by giving a file name to the @option{--sky} option, see
+below. For the rest of this discussion, we'll assume it is already sky
+subtracted.
+
+These three necessary datasets must ofcourse be related: derived from the
+same basic input in steps prior to running Segment. By default, Segment
+will assume they are multiple extensions of a single file, for example see
address@hidden output}. When the Sky-subtracted values are in one file,
+and the detection and Sky standard deviation are in another, you just need
+to use @option{--detection}: in the absence of @option{--std}, Segment will
+look for both the detection labels and Sky standard deviation in the file
+given to @option{--detection}. Ultimately, if all three are in separate
+files, you need to call both @option{--detection} and @option{--std}.
+
+The extensions of the three mandatory inputs can be speicified with
address@hidden, @option{--dhdu}, and @option{--stdhdu}. For a full
+discussion on what to give to these options, see the description of
address@hidden in @ref{Input output options}. To see their default values
+(along with all the other options), run Segment with the
address@hidden (or @option{-P}) option. Just recall that in the
+absence of @option{--detection} and @option{--std}, all three are assumed
+to be in the same file. If you only want to see Segment's default values
+for HDUs on your system, run this command:
+
address@hidden
+$ astsegment -P | grep hdu
address@hidden example
+
+To help in identifying the true clumps, Segment can also work on a
+convolved image (see Section 3.2.1 and Figure 8 of
address@hidden://arxiv.org/abs/1505.01664, Akhlaghi and Ichikawa [2015]}). If
+you have already convolved your input image, you can use the
address@hidden and @option{--chdu} options to specify the file name
+and HDU/extension that it is stored in. Just don't forget that the
+convolved image must also be sky-subtracted before calling
+Segment. However, if a value/file is given to @option{--sky}, the convolved
+values will also be Sky subtracted internally. Alternatively, if you would
+prefer to give a kernel, so Segment does the convolution internally, you
+can use the @option{--kernel} and @option{--khdu}. To disable convolution,
+use @option{--kernel=none}.
+
address@hidden @option
+
address@hidden --sky=STR
+The optional file name containing the Sky values dataset. The extension can
+be specified with @option{--skyhdu}. This dataset must either have the same
+size as the output or the same size as the tessellation (so there is one
+pixel per tile, see @ref{Tessellation}).
+
+When this option is given, its values will be subtraced from the input and
+(the optional) dataset given to @option{--convolved} prior to starting the
+segmentation process.
+
address@hidden --skyhdu
+The HDU/extension containing the Sky value (when @option{--sky} is
+called). For acceptable values, please see the description of
address@hidden in @ref{Input output options}.
+
address@hidden -d STR
address@hidden --detection=STR
+The file name containing the labeled dataset. The extension can be
+specified with @option{--dhdu}. Segmentation (clumps or objects) will only
+take place over the non-zero pixels of this image. The dataset must have
+the same size as the input image. Only datasets with an integer type are
+acceptable for the labeled image, see @ref{Numeric data types}.
+
+If your labeled image only has integer values, but it is stored in a
+floating point container, you can use Gnuastro's Arithmetic program (see
address@hidden) to convert it to an integer container, like the example
+below:
+
address@hidden
+$ astarithmetic float.fits int32 --output=int.fits
address@hidden example
+
address@hidden --dhdu
+The HDU/extension containing the detection labels. For acceptable values,
+please see the description of @option{--hdu} in @ref{Input output options}.
+
address@hidden --std=STR
+The file name containing the Sky standard deviation dataset. The extension
+can be specified with @option{--stdhdu}. This dataset must either have the
+same size as the output or the same size as the tessellation (so there is
+one pixel per tile, see @ref{Tessellation}).
+
address@hidden --stdhdu=INT/STR
+The HDU/extension containing the Sky standard deviation. For acceptable
+values, please see the description of @option{--hdu} in @ref{Input output
+options}.
+
address@hidden -k STR
address@hidden --kernel=STR
+The kernel used to convolve the input image. The usage of this option is
+identical to NoiseChisel's @option{--kernel} option (@ref{General
+NoiseChisel options}). Please see the descriptions there for more.
+
address@hidden --khdu
+The HDU/extension containing the kernel used for convolution. For
+acceptable values, please see the description of @option{--hdu} in
address@hidden output options}.
+
address@hidden --convolved
+The convolved image to avoid internal convolution by Segment. The usage of
+this option is identical to NoiseChisel's @option{--convolved} option
+(@ref{General NoiseChisel options}). Please see the descriptions there for
+more.
+
address@hidden --chdu
+The HDU/extension containing the convolved image (given to
address@hidden). For acceptable values, please see the description of
address@hidden in @ref{Input output options}.
+
address@hidden --rawoutput
+Only output the clumps and objects labeled images. Without this option, the
+first and last extensions of the output will the sky-subtracted dataset and
+the Sky standard deviation dataset (that were inputs to Segment). When the
+datasets are small, this can help in conveniently using the output of
+Segment in @ref{MakeCatalog}. However, when the inputs are large, these
+extra dataset can be burden, and you can use this option to only make the
+raw images and provide the other necessary inputs to MakeCatalog (or any
+other software) with extra options. See @ref{Segment output} for more.
+
address@hidden -L INT[,INT]
address@hidden --largetilesize=INT[,INT]
+The size of the large tiles to use for identifying the clump S/N
+threshold. The usage of this option is identical to NoiseChisel's
address@hidden option (@ref{General NoiseChisel options}). Please
+see the descriptions there for more.
+
address@hidden --continueaftercheck
+Don't abort Segment after producing the check image(s). The usage of this
+option is identical to NoiseChisel's @option{--continueaftercheck} option
+(@ref{General NoiseChisel options}). Please see the descriptions there for
+more.
+
address@hidden table
+
+
address@hidden Segmentation options, Segment output, Segment inputs and general
settings, Invoking astsegment
@subsubsection Segmentation options
-Segmentation is the process of (possibly) breaking up a detection into
-multiple segments (technically called @emph{objects} and @emph{clumps} in
-NoiseChisel). In deep surveys segmentation becomes particularly important
-because we will be detecting more diffuse flux so galaxy images are going
-to overlap more. It is thus very important to be able separate the pixels
-within a detection.
-
-In NoiseChisel, segmentation is done by first finding the `true' clumps
-over a detection and then expanding those clumps to a certain flux
-limit. True clumps are found in a process very similar to the true
-detections explained in @ref{Detection options}, see
address@hidden://arxiv.org/abs/1505.01664, Akhlaghi and Ichikawa [2015]} for
more
-information. If the connections between the grown clumps are weaker than a
-given threshold, the grown clumps are considered to be separate objects.
+The options below can be used to configure every step of the segmentation
+process in the Segment program. For a more complete explanation (with
+figures to demonstrate each step), please see Section 3.2 of
address@hidden://arxiv.org/abs/1505.01664, Akhlaghi and Ichikawa [2015]}, and
+also @ref{Segment}. By default, Segment will follow the procedure described
+in the paper to find the S/N threshold based on the noise properties. This
+can be disabled by directly giving a trustable signal-to-noise ratio to the
address@hidden option.
+
+Recall that you can always see the full list of Gnuastro's options with the
address@hidden (see @ref{Getting help}), or @option{--printparams} (or
address@hidden) to see their values (see @ref{Operating mode options}).
@table @option
address@hidden -B FLT
address@hidden --minskyfrac=FLT
+Minimum fraction (value between 0 and 1) of Sky (undetected) areas in a
+large tile. Only (large) tiles with a fraction of undetected pixels (Sky)
+greater than this value will be used for finding clumps. The clumps found
+in the undetected areas will be used to estimate a S/N threshold for true
+clumps. Therefore this is an important option (to decrease) in crowded
+fields. Operationally, this is almost identical to NoiseChisel's
address@hidden option (@ref{Detection options}). Please see the
+descriptions there for more.
+
@item -m INT
address@hidden --segsnminarea=INT
address@hidden --snminarea=INT
The minimum area which a clump in the undetected regions should have in
order to be considered in the clump Signal to noise ratio measurement. If
this size is set to a small value, the Signal to noise ratio of false
clumps will not be accurately found. It is recommended that this value be
-larger than the value to @option{--detsnminarea}. Because the clumps are
-found on the convolved (smoothed) image while the psudo-detections are
-found on the input image. You can use @option{--checkclumpsn} and
address@hidden to see if your chosen value is reasonable or
-not.
+larger than the value to NoiseChisel's @option{--snminarea}. Because the
+clumps are found on the convolved (smoothed) image while the
+psudo-detections are found on the input image. You can use
address@hidden and @option{--checksegmentation} to see if your chosen
+value is reasonable or not.
address@hidden --checkclumpsn
address@hidden --checksn
Save the S/N values of the clumps into two files ending with
@file{_clumpsn_sky.XXX} and @file{_clumpsn_det.XXX}. The @file{.XXX} is
determined from the @option{--tableformat} option (see @ref{Input output
@@ -14386,13 +14827,24 @@ created. This allows you to inspect the steps leading
to the final S/N
quantile threshold, this behavior can be disabled with
@option{--continueaftercheck}.
address@hidden -g FLT
address@hidden --segquant=FLT
-The quantile of the noise clump Signal to noise ratio distribution. This
-value is used to identify true clumps over the detected regions. You can
-get the full distribution of clumps S/Ns over the undetected areas with the
address@hidden option and see them with
address@hidden
address@hidden --minnumfalse=INT
+The minimum number of clumps over undetected (Sky) regions to identify the
+requested Signal-to-Noise ratio threshold. Operationally, this is almost
+identical to NoiseChisel's @option{--minnumfalse} option (@ref{Detection
+options}). Please see the descriptions there for more.
+
address@hidden -c FLT
address@hidden --snquant=FLT
+The quantile of the signal-to-noise ratio distribution of clumps in
+undetected regions, used to define true clumps. After identifying all the
+usable clumps in the undetected regions of the dataset, the given quantile
+of their signal-to-noise ratios is used to define a the signal-to-noise
+ratio of a ``true'' clump. Effectively, this can be seen as an inverse
+p-value measure. See Figure 9 and Section 3.2.1 of
address@hidden://arxiv.org/abs/1505.01664, Akhlaghi and Ichikawa [2015]} for a
+complete explanation. The full distribution of clump signal-to-noise ratios
+over the undetected areas can be saved into a table with @option{--checksn}
+option and visually inspected with @option{--checksegmentation}.
@item -v
@itemx --keepmaxnearriver
@@ -14405,6 +14857,17 @@ become a very large and with a very high Signal to
noise ratio. In such
cases, the pixel with the maximum flux in the clump will be immediately
touching a river pixel.
address@hidden -s FLT
address@hidden --clumpsnthresh=FLT
+The signal-to-noise threshold for true clumps. If this option is given,
+then the segmentation options above will be ignored and the given value
+will be directly used to identify true clumps over the detections. This can
+be useful if you have a large dataset with similar noise properties. You
+can find a robust signal-to-noise ratio based on a (sufficiently large)
+smaller portion of the dataset. Afterwards, with this option, you can speed
+up the processing on the whole dataset. Other scenarios where this option
+may be useful is when, the image might not contain enough/any Sky regions.
+
@item -G FLT
@itemx --gthresh=FLT
Threshold (multiple of the sky standard deviation added with the sky) to
@@ -14415,15 +14878,14 @@ specified by this option.
@item -y INT
@itemx --minriverlength=INT
The minimum length of a river between two grown clumps for it to be
-considered in Signal to noise ratio estimations. Similar to
address@hidden and @option{--detsnminarea}, if the length of the
-river is too short, the Signal to noise ratio can be noisy and
-unreliable. Any existing rivers shorter than this length will be considered
-as non-existent, independent of their Signal to noise ratio. Since the
-clumps are grown on the input image, this value should best be similar to
-the value of @option{--detsnminarea}. Recall that the clumps were defined
-on the convolved image so @option{--segsnminarea} was larger than
address@hidden
+considered in signal-to-noise ratio estimations. Similar to
address@hidden, if the length of the river is too short, the
+signal-to-noise ratio can be noisy and unreliable. Any existing rivers
+shorter than this length will be considered as non-existent, independent of
+their Signal to noise ratio. The clumps are grown on the input image,
+therefore this value can be smaller than the value given to
address@hidden Recall that the clumps were defined on the convolved
+image so @option{--snminarea} should be larger.
@item -O FLT
@itemx --objbordersn=FLT
@@ -14468,150 +14930,184 @@ created. This behavior can be disabled with
@option{--continueaftercheck}.
@end table
address@hidden Segment output, , Segmentation options, Invoking astsegment
address@hidden Segment output
+
+The main output of Segment are two label datasets (with integer types,
+separating the dataset's elements into different classes) with
+HDU/extension names of @code{CLUMPS} and @code{OBJECTS}. For a complete
+definition of clumps and objects, please see Section 3.2 of
address@hidden://arxiv.org/abs/1505.01664, Akhlaghi and Ichikawa [2015]}.
+
+The clumps are ``true'' local maxima and their surrounding pixels until a
+local minimum (caused by noise fluctuations, or another ``true''
+clump). Therefore it may happen that some of the input detections aren't
+covered by clumps at all (a very diffuse object without a strong central
+peak), while some objects may contain many clumps. Even in those that have
+clumps, there will be regions that are too diffuse. In other words, the
+peaks in those regions are too similar to the peaks in the un-detected
+regions. Those diffuse regions (within the input detected regions) are
+given a negative label (-1) to help you separate them from the undetected
+regions (with a value of zero).
+
+Each clump is labeled respective to its host object. Therefore, if an
+object has three clumps for example, the clumps within it have labels 1, 2
+or 3. As a result, if an initial detected region has multiple objects, each
+with a single clump, all the clumps will have a label of 1. The total
+number of clumps in the dataset is stored in the @code{NCLUMPS} keyword of
+the @code{CLUMPS} extension and printed in the verbose output of Segment.
+
+If the @option{--grownclumps} option is called (or a value of @code{1} is
+given to it in any of the configuration files) the grown clumps will be
+stored in the @code{CLUMPS} extension instead of the original clump
+regions. Note with @option{--grownclumps}, if there is only one clump, or
+no clumps, over a detected region, the whole detected region is given a
+label of 1.
+
+The @code{OBJECTS} extension of the output will give a positive
+counter/label to every positive pixel in the input. As described in
+Akhlaghi and Ichikawa [2015], the clumps are grown and based on the
+signal-to-noise ratio of their connections, they may be considered as part
+of a single object.
+
+The @code{OBJECTS} and @code{CLUMPS} extensions can be used as input into
address@hidden to generate a catalog for higher-level analysis. If you
+want to treat each clump separately, you can give a very large value (or
+even a NaN, which will always fail) to the @option{--gthresh} option (for
+example @code{--gthresh=1e10} or @code{--gthresh=nan}), see
address@hidden options}.
+
address@hidden DS9
address@hidden SAO DS9
+By default, besides the @code{CLUMPS} and @code{OBJECTS} extensions, the
+output will be supplimented with the input dataset and the sky standard
+deviation dataset. This can help in visually inspecting the result when
+viewing the images as a ``Multi-extension data cube'' in SAO DS9 for
+example (see @ref{Viewing multiextension FITS images}). You can simply flip
+through the exetensions and see the same region of the image and its
+corresponding clumps/object labels. It also makes it easy to feed the
+output (as one file) into MakeCatalog when you intend to make a catalog
+afterwards (see @ref{MakeCatalog}.
+
+Therefore, these two default extra extensions of the output are redundant
+(you already had them before calling Segment) and just help in convenience
+when your dataset is small. However, as you start to expand your project to
+larger datasets or more images, the extra volume taken by these extra
+extensions can cause a large burden in your archives and processing. When
+this happens, you can use the @option{--rawoutput} option to avoid such
+excessive waste of precious space (see @ref{Segment inputs and general
+settings}). With this option, the output will only contain the
address@hidden and @code{OBJECTS} datasets. You can then use MakeCatalog's
address@hidden and @option{--clumpsfile} to let it know where to
+find these extensions.
address@hidden NoiseChisel output, , Segmentation options, Invoking
astnoisechisel
address@hidden NoiseChisel output
-
-The default name and directory of the outputs are explained in
address@hidden output}. NoiseChisel's default output (when none of the
-options starting with @option{--check} or the @option{--output} option
-are called) is one file ending with @file{_labeled.fits}. This file
-has the extensions listed below:
-
address@hidden
address@hidden
-A copy of the input image, a copy is placed here for the following
-reasons:
-
address@hidden
-
address@hidden
-By flipping through the extensions, a user can check how accurate the
-detection and segmentation process was.
-
address@hidden
-All the inputs to MakeCatalog (see @ref{MakeCatalog}) are included in
-this one file which makes the running of MakeCatalog after NoiseChisel
-very easy.
-
address@hidden itemize
-
address@hidden
-The object/detection labels. Each pixel in the input image is given a label
-in this extension, the labels start from one. If the
address@hidden option is given, each large connected part of the
-image has one label. Without that option, this extension is going to show
-the labels of the objects that are found after segmentation. The total
-number of labels is stored as the value to the @code{NOBJS}/@code{NDETS}
-keyword in the header of this extension. This number is also printed in
-verbose mode.
-
address@hidden
-The clump labels when @option{--onlydetection} is not called. All the
-pixels in the input image that belong to a true clump are given a positive
-label in this extension. The detected regions that were not a clump are
-given a negative value to clearly identify the sky noise from the diffuse
-detections. The total number of clumps in this image is stored in the
address@hidden keyword of this extension and printed in verbose output.
-
-If the @option{--grownclumps} option is called, or a value of @code{1}
-is given to it in any of the configuration files, then instead of the
-original clump regions, the grown clumps will be stored in this
-extension. Note that if there is only one clump (or no clumps) over a
-detected region, then the whole detected region is given a label of 1.
-
address@hidden
-The final sky value on each pixel. See @ref{Sky value} for a complete
-explanation.
-
address@hidden
-Similar to the previous mesh but for the standard deviation on each
-pixel.
-
address@hidden enumerate
-
-To inspect NoiseChisel's output, you can configure SAO DS9 in your Graphic
-User Interface (GUI) to open NoiseChisel's output as a multi-extension data
-cube. This will allow you to flip through the different extensions and
-visually inspect the results. This process has been described for the GNOME
-GUI (most common GUI in GNU/Linux operating systems) in @ref{Viewing
-multiextension FITS images}.
address@hidden
address@hidden
address@hidden Compression
address@hidden space:} with the @option{--rawoutput}, Segment's output will
+only be two labeled datasets (only containing integers). Since they have no
+noise, such datasets can be compressed very effectively (without any loss
+of data) with exceptionally high compression ratios. You can use the
+following command to compress it with the best ratio:
address@hidden GNU Gzip
address@hidden
+$ gzip --best segment_output.fits
address@hidden example
address@hidden
+The resulting @file{.fits.gz} file can then be fed into any of Gnuastro's
+programs directly, without having to decompress it separately (it will just
+take them a little longer, because they have to decompress it before use).
+
+Alternatively, you can also tell Segment to directly build a
address@hidden output (for example with
address@hidden). But CFITSIO (which reads and writes FITS
+files for Gnuastro) puts more emphasis on compression/decompression
+speed. So it doesn't use the most efficient compression settings (which can
+take slightly longer).
address@hidden cartouche
address@hidden MakeCatalog, Match, NoiseChisel, Data analysis
address@hidden MakeCatalog, Match, Segment, Data analysis
@section MakeCatalog
At the lowest level, a dataset (for example an image) is just a collection
of values, placed after each other in any number of dimensions (for example
an image is a 2D dataset). Each data-element (pixel) just has two
-properties: its position (relative to the rest) and its value. The entire
-input dataset (a large image for example) is rarely treated as a singular
-entity for higher-level address@hidden low-level
-reduction/preparation of a dataset, you do in fact treat a whole image as a
-single entity. You can derive the over-all properties of all the pixels in
-a dataset with Gnuastro's Statistics program (see @ref{Statistics})}. You
-want to know the properties of the scientifically interesting targets that
-are embedded in it. For example the magnitudes, positions and elliptical
-properties of the galaxies that are in the image. MakeCatalog is Gnuastro's
-program to derive higher-level information for @emph{pre-defined} regions
-of a dataset. The role of MakeCatalog in a scientific analysis and the
-benefits of this model of data-analysis (were detection/identification is
-separated from measurement) is discussed in
address@hidden://arxiv.org/abs/1611.06387v1, Akhlaghi [2016]}. We strongly
-recommend reading this short paper for a better understanding of this
-methodology and use MakeCatalog most effectively. However, that paper
-cannot undergo any more change, so this manual is the definitive guide.
-
-As discussed above, you have to define the regions of a dataset that you
-are interested in @emph{before} running MakeCatalog. MakeProfiles currently
-uses labeled dataset(s) for this job. A labeled dataset for a given input
-dataset has the size/dimensions as the input, but its pixels have an
-integer type (see @ref{Numeric data types})@footnote{If the program you
-used to generate the labeled image only outputs floating point types, but
-you know it only has integer valued pixels that are stored in a floating
-point container, you can use Gnuastro's Arithmetic program (see
address@hidden) to change the numerical data type of the image
-(@file{flabel.fits}) to an integer type image (@file{label.fits}) with a
-command like below:@address@hidden astarithmetic flabel.fits int32
---output=label.fits}}: all pixels with the same label (integers larger and
-equal to one) are used to generate the requested output columns of
-MakeCatalog for the row of their labeled value. For example, the flux
-weighted average position of all the pixels with a label of 42 will be
-considered as the central address@hidden @ref{Measuring elliptical
-parameters} for a discussion on this and the derivation of positional
-parameters.} of the 42nd row of the output catalog. Pixels with labels
-equal to or smaller than zero will be ignored by MakeCatalog. In other
-words, the number of rows of the output catalog will be determined from the
-labeled image.
-
-The labeled image maybe created with any address@hidden example, you can
-even use a graphic user interface image editing tool like the GNU Image
-Manipulation Program (or GIMP) and use Gnuastro's ConvertType to convert it
-to a FITS file.}. Within Gnuastro you can use these two solutions depending
-on a-priori/parametric knowledge of the targets you want to study:
+properties: its position (relative to the rest) and its value. In
+higher-level analysis, an entire dataset (an image for example) is rarely
+treated as a singular address@hidden can derive the over-all
+properties of a complete dataset (1D table column, 2D image, or 3D
+datacube) treated as a single entity with Gnuastro's Statistics program
+(see @ref{Statistics}).}. You usually want to know/measure the properties
+of the (separate) scientifically interesting targets that are embedded in
+it. For example the magnitudes, positions and elliptical properties of the
+galaxies that are in the image.
+
+MakeCatalog is Gnuastro's program for localized measurements over a
+dataset. The role of MakeCatalog in a scientific analysis and the benefits
+of this model of data analysis (were detection/segmentation is separated
+from measurement) is discussed in @url{https://arxiv.org/abs/1611.06387v1,
+Akhlaghi [2016]}. We strongly recommend reading this short paper for a
+better understanding of this methodology thus effective usage of
+MakeCatalog, in combination with the other Gnuastro's programs. However,
+that paper cannot undergo any more change, so this manual is the definitive
+guide.
+
+It is important to define your regions of interest @emph{before} running
+MakeCatalog. MakeCatalog is specialized in doing measurements accurately
+and efficiently. Therefore MakeCatalog will not do detection, segmentation,
+or defining apertures on requested positions in your dataset. Following
+Gnuastro's modularity principle, There are separate and higly specialized
+and customizable programs in Gnuastro for these other jobs:
+
@itemize
address@hidden Aperture photometry
address@hidden Photometry, aperture
@item
-When you already know the positions and parametric (for example circular or
-elliptical) properties of the targets, you can use @ref{MakeProfiles} to
-generate a labeled image from another catalog. This is also known as
-aperture photometry (the apertures are defined a-priori).
address@hidden: Detection with a simple threshold.
+
address@hidden
address@hidden: Advanced detection.
+
address@hidden
address@hidden: Segmentation (substructure over detections).
@item
-When the shape of your targets cannot be parameterized accurately (for
-example galaxies), or if you don't know the number/shape of the targets in
-the image, you can use Gnuastro's NoiseChisel program to detect and segment
-(make labeled images of) the @emph{objects} and @emph{clumps} in the input
-image, see @ref{NoiseChisel}.
address@hidden: Aperture creation for known positions.
@end itemize
+These programs will/can return labeled dataset(s) to be fed into
+MakeCatalog. The labeled dataset must have the same size/dimensions as the
+input, but only with integer valued pixels that have the label/counter for
+the feature the pixel belongs to.
+
+These labels are then directly used to make the necessary measurements. For
+example the flux weighted average position of all the pixels with a label
+of 42 will be considered as the central address@hidden
address@hidden elliptical parameters} for a discussion on this and the
+derivation of positional parameters.} of the 42nd row of the output
+catalog. Similarly, the sum of all these pixels will be the 42nd row in the
+brightness column and etc. Pixels with labels equal to or smaller than zero
+will be ignored by MakeCatalog. In other words, the number of rows in
+MakeCatalog's output is already known before running it.
+
+Before getting into the details of running MakeCatalog (in @ref{Invoking
+astmkcatalog}, we'll start with a discussion on the basics of its approach
+to separating detection from measuremens in @ref{Detection and catalog
+production}. A very important factor in any measurement is understanding
+its validity range, or limits. Therefore in @ref{Quantifying measurement
+limits}, we'll discuss how to estimate the reliability of the detection and
+basic measurements. This section will continue with a derivation of
+elliptical parameters from the labelled datasets in @ref{Measuring
+elliptical parameters}. For those who feel MakeCatalog's existing
+measurements/columns aren't enough and would like to add further
+measurements, in @ref{Adding new columns to MakeCatalog}, a checklist of
+steps is provided for readily adding your own new measurements/columns.
+
@menu
* Detection and catalog production:: Discussing why/how to treat these
separately
* Quantifying measurement limits:: For comparing different catalogs.
@@ -14623,28 +15119,50 @@ image, see @ref{NoiseChisel}.
@node Detection and catalog production, Quantifying measurement limits,
MakeCatalog, MakeCatalog
@subsection Detection and catalog production
-As discussed above (@ref{MakeCatalog}), NoiseChisel (Gnuastro's signal
-detection tool, see @ref{NoiseChisel}) does not produce any catalog of the
-detected objects. However, most other common tools in astronomical
-data-analysis (for example
+Most other common tools in low-level astronomical data-analysis (for
+example
address@hidden@url{https://www.astromatic.net/software/sextractor}})
-merge the two processes into one. Gnuastro's modularized methodology is
+merge the two processes of detection and measurement into one. Gnuastro's
+modularized methodology to separating detection from measurements is
therefore new to many experienced astronomers and deserves a short review
here. Further discussion on the benefits of this methodology can be seen in
@url{https://arxiv.org/abs/1611.06387v1, Akhlaghi [2016]}.
-To simplify catalog production from a raw input image in Gnuastro,
-NoiseChisel's output (see @ref{NoiseChisel output}) can be directly fed
-into MakeCatalog. This is good when no further customization is necessary
-and you want a fast/simple. But the modular approach taken by Gnuastro has
-many benefits that will become more apparent as you get more experienced in
-astronomical data analysis and want to be more creative in using your
-valuable data for the exciting scientific project you are working on. In
-short the reasons for this modularity can be classified as below:
+The raw output of any of the programs mentioned in @ref{MakeCatalog} can be
+directly fed into MakeCatalog to get to a fast catalog. This is good when
+no further customization is necessary and you want a fast/simple
+catalog. But the modular approach taken by Gnuastro has many benefits that
+will become clear as you get more experienced in astronomical data analysis
+and want to be more creative in using your valuable data for the exciting
+scientific project you are working on. In short the reasons for this
+modularity can be classified as below:
@itemize
@item
+Simplicity/robustness of independent, modular tools: making a catalog is a
+logically separate process from labeling (detection, segmentation, or
+aperture production). A user might want to do certain operations on the
+labeled regions before creating a catalog for them. Another user might want
+the properties of the same pixels/objects in another image (another filter
+for example) to measure the colors or SED fittings.
+
+Here is an example of doing both: suppose you have images in various
+broad band filters at various resolutions and orientations. The image
+of one color will thus not lie exactly on another or even be in the
+same scale. However, it is imperative that the same pixels be used in
+measuring the colors of galaxies.
+
+To solve the problem, NoiseChisel can be run on the reference image to
+generate the labeled detection image. After wards, the labeled image can be
+warped into the grid of the other color (using @ref{Warp}). MakeCatalog
+will then generate the same catalog for both colors (with the different
+labeled images). It is currently customary to warp the images to the same
+pixel grid, however, modification of the scientific dataset is very harmful
+for the data and creates correlated noise. It is much more accurate to do
+the transformations on the labeled image.
+
address@hidden
Complexity of a monolith: Adding in a catalog functionality to the detector
program will add several more steps (and many more options) to its
processing that can equally well be done outside of it. This makes
@@ -14655,32 +15173,11 @@ As an example, if the parameter you want to measure
over one profile is not
provided by the developers of MakeCatalog. You can simply open this tiny
little program and add your desired calculation easily. This process is
discussed in @ref{Adding new columns to MakeCatalog}. However, if making a
-catalog was part of NoiseChisel, it would require a lot of energy to
-understand all the steps and internal structures of that large program (the
-most complex in Gnuastro) in order to add desired parameter in a catalog.
-
address@hidden
-Simplicity/robustness of independent, modular tools: making a catalog is a
-logically separate process from labeling (detection and segmentation). A
-user might want to do certain operations on the labeled regions before
-creating a catalog for them. Another user might want the properties of the
-same pixels/objects in another image (another filter for example) to
-measure the colors or SED fittings.
-
-Here is an example of doing both: suppose you have images in various
-broad band filters at various resolutions and orientations. The image
-of one color will thus not lie exactly on another or even be in the
-same scale. However, it is imperative that the same pixels be used in
-measuring the colors of galaxies.
-
-To solve the problem, NoiseChisel can be run on the reference image to
-generate the labeled image. After wards, the labeled image can be warped
-into the grid of the other color (using @ref{Warp}). MakeCatalog will then
-generate the same catalog for both colors (with the different labeled
-images). It is currently customary to warp the images to the same pixel
-grid, however, modification of the scientific dataset is very harmful for
-the data and creates correlated noise. It is much more accurate to do the
-transformations on the labeled image.
+catalog was part of NoiseChisel for example, adding a new
+column/measurement would require a lot of energy to understand all the
+steps and internal structures of that huge program. It might even be so
+intertwined with its processing, that adding new columns might cause
+problems/bugs in its primary job (detection).
@end itemize
@@ -14922,10 +15419,10 @@ today, we can make customized segmentation maps for
each object.
If requested, MakeCatalog will estimate teh the upper limit magnitude is
found for each object in the image separately, the procedure is fully
-configurable with the options in @ref{Upper-limit magnitude settings}. If
-one value for the whole image is required, you can either use the surface
-brightness limit above or make a circular aperture and feed it into
-MakeCatalog to request an upper-limit magnitude for it.
+configurable with the options in @ref{Upper-limit settings}. If one value
+for the whole image is required, you can either use the surface brightness
+limit above or make a circular aperture and feed it into MakeCatalog to
+request an upper-limit magnitude for it.
@end table
@@ -15147,6 +15644,14 @@ column requires new raw calculations, add a row to the
respective list. If
your calculation requires any other settings parameters, you should add a
variable to the @code{mkcatalogparams} structure.
address@hidden ui.c
+If the new column needs raw calculations (an entry was added in
address@hidden and @code{clumpcols}), specify which inputs it needs in
address@hidden, similar to the other options. Afterwards, if
+your column includes any particular settings (you needed to add a variable
+to the @code{mkcatalogparams} structure in @file{main.h}), you should do
+the sanity checks and preparations for it here.
+
@item ui.h
The @code{option_keys_enum} associates a unique value for each option to
MakeCatalog. The options that have a short option version, the single
@@ -15154,6 +15659,7 @@ character short comment is used for the value. Those
that don't have a
short option version, get a large integer automatically. You should add a
variable here to identify your desired column.
+
@cindex GNU C library
@item args.h
This file specifies all the parameters for the GNU C library, Argp
@@ -15162,11 +15668,6 @@ new column, just copy an existing set of parameters
and change the first,
second and 5th values (the only ones that differ between all the columns),
you should use the macro you defined in @file{ui.h} here.
address@hidden ui.c
-If your column includes any particular settings (you added a variable to
-the @code{mkcatalogparams} structure in @file{main.h}), you should do the
-sanity checks and preparations for it here. Otherwise, you can ignore this
-file.
@item columns.c
This file contains the main definition and high-level calculation of your
@@ -15201,9 +15702,9 @@ Update this manual and add a description for the new
column.
@node Invoking astmkcatalog, , Adding new columns to MakeCatalog, MakeCatalog
@subsection Invoking MakeCatalog
-MakeCatalog will make a catalog from an input image and at least on labeled
-image. The executable name is @file{astmkcatalog} with the following
-general template
+MakeCatalog will do measurements and produce a catalog from a labeled
+dataset and optional values dataset(s). The executable name is
address@hidden with the following general template
@example
$ astmkcatalog [OPTION ...] InputImage.fits
@@ -15214,137 +15715,182 @@ One line examples:
@example
## Create catalog with RA, Dec, Magnitude and Magnitude error,
-## `input.fits' is NoiseChisel's output:
-$ astmkcatalog --ra --dec --magnitude --magnitudeerr input.fits
+## from Segment's output:
+$ astmkcatalog --ra --dec --magnitude --magnitudeerr seg-out.fits
## Same catalog as above (using short options):
-$ asmkcatalog -rdmG input.fits
+$ asmkcatalog -rdmG seg-out.fits
-## Write the catalog to a FITS table:
-$ astmkcatalog -mpQ --output=cat.fits input_labeled.fits
+## Write the catalog to a text table:
+$ astmkcatalog -mpQ input_labeled.fits --output=cat.txt
## Read the columns to create from `columns.conf':
$ astmkcatalog --config=columns.conf input_labeled.fits
-## Use different images for the objects and clumps inputs:
-$ astmkcatalog --objectsfile=K_labeled.fits --objectshdu=1 \
- --clumpsfile=K_labeled.fits --clumpshdu=2 i_band.fits
+## Use object and clump labels from a K-band image, but pixel values
+## from an i-band image.
+$ astmkcatalog K_labeled.fits --hdu=1 --clumpscat \
+ --clumpsfile=K_labeled.fits --clumpshdu=2 \
+ --valuesfile=i_band.fits
@end example
@cindex Gaussian
@noindent
-If MakeCatalog is to do processing, an input image should be provided with
-the recognized extensions as input data, see @ref{Arguments}. The options
-described in this section are those that are only particular to
-MakeProfiles. For operations that MakeProfiles shares with other programs
-(mainly involving input/output or general processing steps), see
address@hidden options}. Also see @ref{Common program behavior} for some
-general characteristics of all Gnuastro programs including MakeCatalog.
-
-MakeCatalog needs 4 (or 5) images as input. These images can be separate
-extensions in one file (NoiseChisel's default output), or each can have its
-own file and its own extension. See @ref{NoiseChisel output} for the
-list. The clump labels image is not mandatory (when no clump catalog is
-required, for example in aperture photometry). When inspecting the object
-labels image, MakeProfiles will look for a @code{WCLUMPS} (short for
-with-clumps) header keyword. If that keyword is present and has a value of
address@hidden, @code{1}, or @code{y} (case insensitive) then a clump image
-must also be provided and a clump catalog will be made. When @code{WCLUMPS}
-isn't present or has any other value, only an object catalog will be
-created and all clump related options/columns will be ignored.
-
address@hidden Photometry, aperture
address@hidden Aperture photometry
-For example, if you only need an object catalog from NoiseChisel's output,
-you can use Gnuastro's Fits program (see @ref{Fits}) to modify or remove
-the @code{WCLUMPS} keyword in the objects HDU, then run MakeCatalog on
-it. Another example can be aperture photometry: let's assume you have made
-your labeled image (defining the apertures) with MakeProfiles. Clumps are
-not defined in this context, so besides the input and labeled image, you
-only need NoiseChisel's Sky and Sky standard deviation images (run
-NoiseChisel with the @option{--onlydetection} option). Since MakeProfile's
-output doesn't contain the @code{WCLUMPS} keyword, you just have to specify
-your labeled image with the @option{--objectsfile} option and also set its
-HDU. Note that labeled images have to be an integer type. Therefore, if you
-are using MakeProfiles to define the apertures/labels, you can use its
address@hidden for example, see @ref{Input output options} and
address@hidden data types}.
+If MakeCatalog is to do processing (not printing help or option values), an
+input labeled image should be provided. The options described in this
+section are those that are particular to MakeProfiles. For operations that
+MakeProfiles shares with other programs (mainly involving input/output or
+general processing steps), see @ref{Common options}. Also see @ref{Common
+program behavior} for some general characteristics of all Gnuastro programs
+including MakeCatalog.
+
+The various measurements/columns of MakeCatalog are requested as options,
+either on the command-line or in configuration files, see
address@hidden files}. The full list of available columns is available
+in @ref{MakeCatalog output columns}. Depending on the requested columns,
+MakeCatalog needs more than one input dataset, for more details, please see
address@hidden inputs and basic settings}. The upper-limit measurements
+in particular need several configuration options which are thoroughly
+discussed in @ref{Upper-limit settings}. Finally, in @ref{MakeCatalog
+output file} the output file(s) created by MakeCatalog are discussed.
-When a clump catalog is also desired, two catalogs will be made: one for
-the objects (suffixed with @file{_o.txt} or @file{_o.fits}) and another for
-the clumps (suffixed with @file{_c.txt} or @file{_c.fits}). Therefore if
-any value is given to the @option{--output} option, MakeCatalogs will
-replace these two suffixes with any existing suffix in the given value. If
-no output value is given, MakeCatalog will use the input name, see
address@hidden output}. The format of the output table is specified with
-the @option{--tableformat} option, see @ref{Input output options}.
address@hidden
+* MakeCatalog inputs and basic settings:: Input files and basic settings.
+* Upper-limit settings:: Settings for upper-limit measurements.
+* MakeCatalog output columns:: Available columns in MakeCatalog's output
table.
+* MakeCatalog output file:: File names of MakeCatalog's output table.
address@hidden menu
-When MakeCatalog is run on multiple threads, the clumps catalog rows will
-not be sorted by object since each object is processed independently by one
-thread and threaded applications are asynchronous. The clumps in each
-object will be sorted based on their labels, but you will find lower-index
-objects come after higher-index ones (especially if they have more clumps
-and thus take more time). If the order is very important for you, you can
-run the following command to sort the rows by object ID (and clump ID with
-each object):
address@hidden MakeCatalog inputs and basic settings, Upper-limit settings,
Invoking astmkcatalog, Invoking astmkcatalog
address@hidden MakeCatalog inputs and basic settings
+
+MakeCatalog works by using a localized/labeled dataset (see
address@hidden). This dataset maps/labels pixels to a specific target
+(row number in the final catalog) and is thus the only necessary input
+dataset to produce a minimal catalog in any situation. Because it only has
+labels/counters, it must have an integer type (see @ref{Numeric data
+types}), see below if your labels are in a floating point container. When
+the requested measurements only need this dataset (for example
address@hidden, @option{--geoy}, or @option{--geoarea}), MakeCatalog won't
+read any more datasets.
+
+Low-level measurements that only use the labeled image are rarely
+sufficient for any high-level science case. Therefore necessary input
+datasets depend on the requested columns in each run. For example, let's
+assume you want the brightness/magnitude and signal-to-noise ratio of your
+labeled regions. For these columns, you will also need to provide an extra
+dataset containing values for every pixel of the labeled input (to measure
+brightness) and another for the Sky standard deviation (to measure
+error). All such auxiliary input files have to have the same size (number
+of pixels in each dimension) as the input labeled image. Their numeric data
+type is irrelevant (they will be converted to 32-bit floating point
+internally). For the full list of available measurements, see
address@hidden output columns}.
+
+The ``values'' dataset is used for measurements like brightness/magnitude,
+or flux-weighted positions. If it is a real image, by default it is assumed
+to be already Sky-subtracted prior to running MakeCatalog. If it isn't, you
+use the @option{--subtractsky} option to, so MakeCatalog reads and
+subtracts the Sky dataset before any processing. To obtain the Sky value,
+you can use the @option{--sky} option of @ref{Statistics}, but the best
+recommended method is @ref{NoiseChisel}, see @ref{Sky value}.
+
+MakeCatalog can also do measurements on sub-structures of detections. In
+other words, it can produce two catalogs. Following the nomenclature of
+Segment (see @ref{Segment}), the main labeled input dataset is known as
+``object'' labels and the (optional) sub-structure input dataset is known
+as ``clumps''. If MakeCatalog is run with the @option{--clumpscat} option,
+it will also need a labeled image containing clumps, similar to what
+Segment produces (see @ref{Segment output}). Since clumps are defined
+within detected regions (they exist over signal, not noise), MakeCatalog
+uses their boundaries to subtract the level of signal under them.
+
+There are separate options to explicitly request a file name and
+HDU/extension for each of the required input datasets as fully described
+below (with the @option{--*file} format). When each dataset is in a
+separate file, these options are necessary. However, one great advantage of
+the FITS file format, that is heavily used in astronomy, is that it allows
+the storage of multiple datasets in one file. So in most situations (for
+example if you are using the outputs of @ref{NoiseChisel} or
address@hidden), all the necessary input datasets can be in one file.
+
+When none of the @option{--*file} options are given, MakeCatalog will
+assume the necessary input datasets are in the file given as its argument
+(without any option). When the Sky or Sky standard deviation datasets are
+necessary and the only @option{--*file} option called is
address@hidden, MakeCatalog will search for these datasets (with the
+default/given HDUs) in the file given to @option{--valuesfile} (before
+looking into the the main argument file).
+
+It may happen that your labeled objects image was created with a program
+that only outputs floating point files. However, you know it only has
+integer valued pixels that are stored in a floating point container. In
+such cases, you can use Gnuastro's Arithmetic program (see
address@hidden) to change the numerical data type of the image
+(@file{float.fits}) to an integer type image (@file{int.fits}) with a
+command like below:
@example
-$ awk '!/^#/' out_c.txt | sort -g -k1,1 -k2,2
address@hidden astarithmetic float.fits int32 --output=int.fits}
@end example
address@hidden
-* MakeCatalog input files:: Specifying the different input files.
-* MakeCatalog general settings:: Options for general column settings.
-* Upper-limit magnitude settings:: Necessary to define upper-limit magnitudes.
-* MakeCatalog output columns:: How to specify the columns in the output.
address@hidden menu
-
address@hidden MakeCatalog input files, MakeCatalog general settings, Invoking
astmkcatalog, Invoking astmkcatalog
address@hidden MakeCatalog input files
-
-MakeCatalog needs multiple images as input: a values image, one (or two)
-labeled images and Sky and Sky standard deviation images. The options
-described in this section allow you to identify them. If you use the
-default output of NoiseChisel (see @ref{NoiseChisel output}) you don't have
-to worry about any of these options and just give NoiseChisel's output file
-to MakeCatalog as described in @ref{Invoking astmkcatalog}.
+To summarize: if the input file to MakeCatalog is the default/full output
+of Segment (see @ref{Segment output}) you don't have to worry about any of
+the @option{--*file} options below. You can just give Segment's output file
+to MakeCatalog as described in @ref{Invoking astmkcatalog}. To feed
+NoiseChisel's output into MakeCatalog, just change the labeled dataset's
+header (with @option{--hdu=DETECTIONS}). The full list of input dataset
+options and general setting options are described below.
@table @option
address@hidden -O STR
address@hidden --objectsfile=STR
-The file name of the object labels image, if the image is in another
-extension of the input file, calling this option is not mandatory, just
-specify the extension/HDU with the @option{--objectshdu} option.
-
address@hidden --objectshdu=STR
-The HDU/extension of the object labels image. Only pixels with values above
-zero will be considered. The objects label image has to be an integer data
-type (see @ref{Numeric data types}) and only pixels with a value larger
-than zero will be used. If this extension contains the @code{WCLUMPS}
-keyword with a value of @code{yes}, @code{1}, or @code{y} (not case
-sensitive), then MakeCatalog will also build a clumps catalog, see
address@hidden astmkcatalog}.
address@hidden -v STR
address@hidden --valuesfile=STR
+The file name of the (sky-subtracted) values dataset. When any of the
+columns need values to associate with the input labels (for example to
+measure the brightness/magnitude of a galaxy), MakeCatalog will look into a
+``values'' for the respective pixel values. In most common processing, this
+is the actual astronomical image that the labels were defined, or detected,
+over. The HDU/extension of this dataset in the given file can be specified
+with @option{--valueshdu}. If this option is not called, MakeCatalog will
+look for the given extension in the main input file.
+
address@hidden --valueshdu=STR/INT
+The name or number (counting from zero) of the extension containing the
+``values'' dataset, see the descriptions above and those in
address@hidden for more.
address@hidden -C STR
address@hidden -l STR
@itemx --clumpsfile=STR
-Similar to @option{--objlabs} but for the labels of the clumps. This is
-only necessary if the image containing clump labels is not in the input
-file and the objects image has a @code{WCLUMPS} keyword, see
address@hidden
+The file containing the labeled clumps dataset when @option{--clumpscat} is
+called. For more, see the description of @option{--clumpscat}. When
address@hidden is called, but this option isn't, MakeCatalog will
+look into the main input file (given as an argument) for the required
+(@option{--clumpshdu}).
@item --clumpshdu=STR
-The HDU/extension of the object labels image. Only pixels with values above
-zero will be considered. The objects label image has to be an integer data
-type (see @ref{Numeric data types}) and only pixels with a value larger
-than zero will be used.
+The HDU/extension of the clump labels dataset. Only pixels with values
+above zero will be considered. The clump lables dataset has to be an
+integer data type (see @ref{Numeric data types}) and only pixels with a
+value larger than zero will be used. See @ref{Segment output} for a
+description of the expected format.
@item -s STR
@itemx --skyfile=STR
-File name of an image keeping the Sky value for each pixel.
+File name of dataset keeping the Sky value for each pixel. The Sky dataset
+is only necessary when @option{--subtractsky} is called or when a column
+directly related to the Sky value is requested (currently
address@hidden). When the Sky dataset is necessary and this option is not
+called, MakeCatalog will first look into the @option{--valuesfile} (if it
+is given) and then the main input file (given as an argument). By default
+the values dataset is assumed to be already Sky subtracted, so this dataset
+is not necessary for many of the columns.
+
+This dataset may be a tessellation values dataset with one element per tile
+(see @option{--oneelempertile} of @ref{Processing options}).
@item --skyhdu=STR
-The HDU of the Sky value image.
+HDU/extension of the Sky dataset, see @option{--skyfile}.
@item -t STR
@itemx --stdfile=STR
@@ -15364,81 +15910,47 @@ measurements will be over-estimated.
@item --stdhdu=STR
The HDU of the Sky value standard deviation image.
address@hidden --variance
-The image given to @option{--stdfile} has the variance of every pixel, not
-standard deviation.
-
address@hidden table
-
-
-
address@hidden MakeCatalog general settings, Upper-limit magnitude settings,
MakeCatalog input files, Invoking astmkcatalog
address@hidden MakeCatalog general settings
-
-Some of the columns require particular settings (for example the zero point
-magnitude for measuring magnitudes), the options in this section can be
-used for such configurations.
-
address@hidden @option
-
@item -z FLT
@itemx --zeropoint=FLT
The zero point magnitude for the input image, see @ref{Flux Brightness and
magnitude}.
address@hidden -E
address@hidden --skysubtracted
-If the image has already been sky subtracted by another program, then you
-need to notify MakeCatalog through this option. Note that this is only
-relevant when the Signal to noise ratio is to be calculated.
-
address@hidden -T FLT
address@hidden --threshold=FLT
-For all the columns, only consider pixels that are above a given relative
-threshold. Symbolizing the value of this option as @mymath{T}, the Sky for
-a pixel at @mymath{(i,j)} with @mymath{\mu_{ij}} and its Standard deviation
-with @mymath{\sigma_{ij}}, that pixel will only be used if its value
-(@mymath{B_{ij}}) satisfies this condition:
address@hidden>\mu_{ij}+{T}\sigma_{ij}}. The only calculations that will
-not be affected are is the average river values (@option{--riverave}),
-since they are used as a reference. A commented row will be added in the
-header of the output catalog that will print the given value, since this is
-a very important issue, it starts with @command{**IMPORTANT**}.
-
-NoiseChisel will detect very diffuse signal which is useful in most
-cases where the aggregate properties of the detections are desired,
-since there is signal there (with the desired certainty). However, in
-some cases, only the properties of the peaks of the objects/clumps are
-desired, for example in attempting to separate stars from galaxies,
-the peaks are the major target and the diffuse regions only act to
-complicate the separation. With this option, MakeCatalog will simply
-ignore any pixel below the relative threshold.
address@hidden --variance
+The dataset given to @option{--stdfile} (and @option{--stdhdu} has the Sky
+variance of every pixel, not the Sky standard deviation.
address@hidden NaN
-This option is not mandatory, so if it isn't given (after reading the
-command-line and all configuration files, see @ref{Configuration
-files}), MakeCatalog will still operate. However, if it has a value in
-any lower-level configuration file and you want to ignore that value
-for this particular run or in a higher-level configuration file, then
-set it to NaN, for example @option{--threshold=nan}. Gnuastro uses the
-C library's @code{strtod} function to read floats, which is not
-case-sensitive in reading NaN values. But to be consistent, it is good
-practice to only use @option{nan}.
-
address@hidden --nsigmag=FLT
-The median standard deviation (from the standard deviation image) will be
-multiplied by the value to this option and its magnitude will be reported
-in the comments of the output catalog. This value is a per-pixel value, not
-per object/clump and is not found over an area or aperture, like the common
address@hidden values that are commonly reported as a measure of depth or
-the upper-limit measurements (see @ref{Quantifying measurement
-limits}).
address@hidden -C
address@hidden --clumpscat
+Produce a clumps catalog also. When this option is given, MakeCatalog will
+also look for secondary (sub-structure) labeled dataset and produce a
+catalog from that. If @option{--clumpsfile} isn't given, it will look into
+the main input argument/file for the HDU/extension containing the
+dataset. The extension can be specified with @option{--clumpshdu}. For more
+on the definition of ``clumps'', see @ref{Segment}.
+
address@hidden --sfmagnsigma=FLT
+The median standard deviation (from a @command{MEDSTD} keyword in the Sky
+standard deviation image) will be multiplied by the value to this option
+and its magnitude will be reported in the comments of the output
+catalog. This value is a per-pixel value, not per object/clump and is not
+found over an area or aperture, like the common @mymath{5\sigma} values
+that are commonly reported as a measure of depth or the upper-limit
+measurements (see @ref{Quantifying measurement limits}).
+
address@hidden --sfmagarea=FLT
+Area (in arcseconds squared) to convert the per-pixel estimation of
address@hidden in the comments section of the output tables. Note
+that this is just a unit conversion using the World Coordinate System (WCS)
+information in the input's header. It does not actually do any measurements
+on this area. For random measurements on any area, please use the
+upper-limit columns of MakeCatalog (see the discussion on upper-limit
+measurements in @ref{Quantifying measurement limits}).
@end table
address@hidden Upper-limit magnitude settings, MakeCatalog output columns,
MakeCatalog general settings, Invoking astmkcatalog
address@hidden Upper-limit magnitude settings
address@hidden Upper-limit settings, MakeCatalog output columns, MakeCatalog
inputs and basic settings, Invoking astmkcatalog
address@hidden Upper-limit settings
The upper limit magnitude was discussed in @ref{Quantifying measurement
@@ -15544,10 +16056,27 @@ The multiple of the final (@mymath{\sigma}-clipped)
standard deviation (or
@mymath{\sigma}) used to measure the upper-limit brightness or
magnitude.
address@hidden --checkupperlimit=INT[,INT]
+Print a table of positions and measured values for all the full random
+distribution used for one particular object or clump. If only one integer
+is given to this option, it is interpretted to be an object's label. If two
+values are given, the first is the object label and the second is the ID of
+requested clump wihtin it.
+
+The output is a table with three columns (its type is determined with the
address@hidden option, see @ref{Input output options}). The first
+two columns are the position of the first pixel in each random sampling of
+this particular object/clump. The the third column is the measured flux
+over that region. If the region overlapped with a detection or masked
+pixel, then its measured value will be a NaN (not-a-number). The total
+number of rows is thus un-known, but you can be sure that the number of
+rows with non-NaN measurements is the number given to the @option{--upnum}
+option.
+
@end table
address@hidden MakeCatalog output columns, , Upper-limit magnitude settings,
Invoking astmkcatalog
address@hidden MakeCatalog output columns, MakeCatalog output file, Upper-limit
settings, Invoking astmkcatalog
@subsubsection MakeCatalog output columns
The final group of options particular to MakeCatalog are those that specify
@@ -15580,22 +16109,6 @@ if only object catalogs are required, it has the same
effect as
@item --idinhostobj
[Clumps] The ID of this clump in its host object.
address@hidden -C
address@hidden --numclumps
-[Objects] The number of clumps in this object.
-
address@hidden -a
address@hidden --area
-The raw area (number of pixels) in any clump or object independent of what
-pixel it lies over (if it is NaN/blank or unused for example).
-
address@hidden --clumpsarea
-[Objects] The total area of all the clumps in this object.
-
address@hidden --weightarea
-The area (number of pixels) used in the flux weighted position
-calculations.
-
@item -x
@itemx --x
The flux weighted center of all objects and clumps along the first FITS
@@ -15708,6 +16221,10 @@ If no usable pixels (blank or below the threshold) are
present over the
clump or object, the stored value will be NaN (note that zero is
meaningful).
address@hidden --brightnesserr
+The (@mymath{1\sigma}) error in measuring the brightness of objects or
+clumps.
+
@item --clumpbrightness
[Objects] The total brightness of the clumps within an object. This is
simply the sum of the pixels associated with clumps in the object. If no
@@ -15715,16 +16232,6 @@ usable pixels (blank or below the threshold) are
present over the clump or
object, the stored value will be NaN, because zero (note that zero is
meaningful).
address@hidden --mean
-The mean sky subtracted value of pixels within the object or clump. For
-clumps, the average river flux is subtracted from the sky subtracted
-mean.
-
address@hidden --median
-The median sky subtracted value of pixels within the object or clump. For
-clumps, the average river flux is subtracted from the sky subtracted
-median.
-
@item --noriverbrightness
[Clumps] The Sky (not river) subtracted clump brightness. By definition,
for the clumps, the average brightness of the rivers surrounding it are
@@ -15748,6 +16255,16 @@ If no usable pixels (blank or below the possibly given
threshold) are
present over the clump or object, the stored value will be NaN (note that
zero is meaningful).
address@hidden --mean
+The mean sky subtracted value of pixels within the object or clump. For
+clumps, the average river flux is subtracted from the sky subtracted
+mean.
+
address@hidden --median
+The median sky subtracted value of pixels within the object or clump. For
+clumps, the average river flux is subtracted from the sky subtracted
+median.
+
@item -m
@itemx --magnitude
The magnitude of clumps or objects, see @option{--brightness}.
@@ -15771,20 +16288,20 @@ defined for work on adding these sources of error too.
@item --upperlimit
The upper limit value (in units of the input image) for this object or
clump. See @ref{Quantifying measurement limits} and @ref{Upper-limit
-magnitude settings} for a complete explanation. This is very important for
-the fainter and smaller objects in the image where the measured magnitudes
-are not reliable.
+settings} for a complete explanation. This is very important for the
+fainter and smaller objects in the image where the measured magnitudes are
+not reliable.
@item --upperlimitmag
The upper limit magnitude for this object or clump. See @ref{Quantifying
-measurement limits} and @ref{Upper-limit magnitude settings} for a complete
+measurement limits} and @ref{Upper-limit settings} for a complete
explanation. This is very important for the fainter and smaller objects in
the image where the measured magnitudes are not reliable.
@item --upperlimitonesigma
The @mymath{1\sigma} upper limit value (in units of the input image) for
this object or clump. See @ref{Quantifying measurement limits} and
address@hidden magnitude settings} for a complete explanation. When
address@hidden settings} for a complete explanation. When
@option{--upnsigma=1}, this column's values will be the same as
@option{--upperlimit}.
@@ -15792,16 +16309,15 @@ this object or clump. See @ref{Quantifying
measurement limits} and
The position of the total brightness measured within the distribution of
randomly placed upperlimit measurements in units of the distribution's
@mymath{\sigma} or standard deviation. See @ref{Quantifying measurement
-limits} and @ref{Upper-limit magnitude settings} for a complete
-explanation.
+limits} and @ref{Upper-limit settings} for a complete explanation.
@item --upperlimitquantile
The position of the total brightness measured within the distribution of
randomly placed upperlimit measurements as a quantile (value between 0 or
-1). See @ref{Quantifying measurement limits} and @ref{Upper-limit magnitude
-settings} for a complete explanation. If the object is brighter than the
-brightest randomly placed profile, a value of @code{inf} is returned. If it
-is less than the minimum, a value of @code{-inf} is reported.
+1). See @ref{Quantifying measurement limits} and @ref{Upper-limit settings}
+for a complete explanation. If the object is brighter than the brightest
+randomly placed profile, a value of @code{inf} is returned. If it is less
+than the minimum, a value of @code{-inf} is reported.
@item --upperlimitskew
@cindex Skewness
@@ -15848,6 +16364,28 @@ The sky value standard deviation (per pixel) for this
clump or
object. Like @option{--sky}, this is the average of the values in the
input sky standard deviation image pixels that lie over this object.
address@hidden -C
address@hidden --numclumps
+[Objects] The number of clumps in this object.
+
address@hidden -a
address@hidden --area
+The raw area (number of pixels) in any clump or object independent of what
+pixel it lies over (if it is NaN/blank or unused for example).
+
address@hidden --clumpsarea
+[Objects] The total area of all the clumps in this object.
+
address@hidden --weightarea
+The area (number of pixels) used in the flux weighted position
+calculations.
+
address@hidden --geoarea
+The area of all the pixels labeled with an object or clump. Note that
+unlike @option{--area}, pixel values are completely ignored in this
+column. For example, if a pixel value is blank, it won't be counted in
address@hidden, but will be counted here.
+
@item -A
@itemx --semimajor
The pixel-value weighted semi-major axis of the profile (assuming it is an
@@ -15858,6 +16396,10 @@ ellipse) in units of pixels. See @ref{Measuring
elliptical parameters}.
The pixel-value weighted semi-minor axis of the profile (assuming it is an
ellipse) in units of pixels. See @ref{Measuring elliptical parameters}.
address@hidden --axisratio
+The pixel-value weighted axis ratio (semi-minor/semi-major) of the object
+or clump.
+
@item -p
@itemx --positionangle
The pixel-value weighted angle of the semi-major axis with the first FITS
@@ -15871,6 +16413,10 @@ assuming it is an ellipse.
The geometric (ignoring pixel values) semi-minor axis of the profile,
assuming it is an ellipse.
address@hidden --geoaxisratio
+The geometric (ignoring pixel values) axis ratio of the profile, assuming
+it is an ellipse.
+
@item --geopositionangle
The geometric (ignoring pixel values) angle of the semi-major axis
with the first FITS axis in degrees.
@@ -15880,6 +16426,34 @@ with the first FITS axis in degrees.
address@hidden MakeCatalog output file, , MakeCatalog output columns, Invoking
astmkcatalog
address@hidden MakeCatalog output file
+
+When a clump catalog is also desired, two catalogs will be made: one for
+the objects (suffixed with @file{_o.txt} or @file{_o.fits}) and another for
+the clumps (suffixed with @file{_c.txt} or @file{_c.fits}). Therefore if
+any value is given to the @option{--output} option, MakeCatalogs will
+replace these two suffixes with any existing suffix in the given value. If
+no output value is given, MakeCatalog will use the input name, see
address@hidden output}. The format of the output table is specified with
+the @option{--tableformat} option, see @ref{Input output options}.
+
+When MakeCatalog is run on multiple threads, the clumps catalog rows will
+not be sorted by object since each object is processed independently by one
+thread and threaded applications are asynchronous. The clumps in each
+object will be sorted based on their labels, but you will find lower-index
+objects come after higher-index ones (especially if they have more clumps
+and thus take more time). If the order is very important for you, you can
+run the following command to sort the rows by object ID (and clump ID with
+each object):
+
address@hidden
+$ awk '!/^#/' out_c.txt | sort -g -k1,1 -k2,2
address@hidden example
+
+
+
+
@node Match, , MakeCatalog, Data analysis
@section Match
@@ -24287,8 +24861,8 @@ applied on the dataset, then @code{num} dilations.
@deftypefun size_t gal_binary_connected_components (gal_data_t @code{*binary},
gal_data_t @code{**out}, int @code{connectivity})
address@hidden Connected components
@cindex Breadth first search
address@hidden Connected component labeling
Return the number of connected components in @code{binary} through the
breadth first search algorithm (finding all pixels belonging to one
component before going on to the next). Connection between two pixels is
@@ -25678,13 +26252,17 @@ use all the great features that Gnuastro has already
defined for such
operations.
To make it easier to learn/apply the internal program infra-structure
-discussed in @ref{Mandatory source code files}, Gnuastro ships with a
-template program when using the @ref{Version controlled source}. It is not
-available in the Gnuastro tarball so it doesn't confuse people using the
-tarball. The @file{bin/TEMPLATE} directory in Gnuastro's clone contains the
-bare-minimum files necessary to define a new program and all the necessary
-files/functions are pre-defined there. You can take the following steps if
-you want to add a new program to Gnuastro:
+discussed in @ref{Mandatory source code files}, in the @ref{Version
+controlled source}, Gnuastro ships with a template program . This template
+program is not available in the Gnuastro tarball so it doesn't confuse
+people using the tarball. The @file{bin/TEMPLATE} directory in Gnuastro's
+Git repository contains the bare-minimum files necessary to define a new
+program and all the basic/necessary files/functions are pre-defined
+there.
+
+Below you can see a list of initial steps to take for customizing this
+template. We just assume that after cloning Gnuastro's history, you have
+already bootstrapped Gnuastro, if not, please see @ref{Bootstrapping}.
@enumerate
@item
@@ -25697,32 +26275,84 @@ $ cp -R bin/TEMPLATE bin/myprog
@end example
@item
+As with all source files in Gnuastro, all the files in template also have a
+copyright notice at their top. Open all the files and correct these
+notices: 1) The first line contains a single-line description of the
+program. 2) In the second line only the name or your program needs to be
+fixed and 3) Add your name and email as a ``Contributing author''. As your
+program grows, you will need to add new files, don't forget to add this
+notice in those new files too, just put your name and email under
+``Original author'' and correct the copyright years.
+
address@hidden
Open @file{configure.ac} in the top Gnuastro source. This file manages the
operations that are done when a user runs @file{./configure}. Going down
-the file, you will notice repetitive parts for each program. Copy one of
-those and correct the names of the copied program to your new program
-name. We follow alphabetic ordering here, so please place it
-correctly. There are multiple places where this has to be done, so be
+the file, you will notice repetitive parts for each program. You will
+notice that the program names follow an alphabetic ordering in each
+part. There is also a commented line/patch for the @file{TEMPLATE} program
+in each part. You can copy one line/patch (from the program above or below
+your desired name for example) and paste it in the proper place for your
+new program. Then correct the names of the copied program to your new
+program name. There are multiple places where this has to be done, so be
patient and go down to the bottom of the file. Ultimately add
@file{bin/myprog/Makefile} to @code{AC_CONFIG_FILES}, only here the
-ordering depends on the length of the name.
+ordering depends on the length of the name (it isn't alphabetical).
@item
Open @file{Makefile.am} in the top Gnuastro source. Similar to the previous
-step, add your new program similar to all the other programs.
+step, add your new program similar to all the other programs. Here there
+are only two places: 1) at the top where we define the conditionals (three
+lines per program), and 2) immediately under it as part of the value for
address@hidden
+
address@hidden
+Open @file{doc/Makefile.am} and similar to @file{Makefile.am} (above), add
+the proper entries for the man-page of your program to be created (here,
+the variable that keeps all the man-pages to be created is
address@hidden). Then scroll down and add a rule to build the
+man-page similar to the other existing rules (in alphabetical order). Don't
+forget to add a short one-line description here, it will be displayed on
+top of the man-page.
@item
-Change @code{TEMPLATE} to @code{myprog} in the file names and contents of
-the files in the @file{bin/myprog/} directory.
+Change @code{TEMPLATE.c} and @code{TEMPLATE.h} to @code{myprog.c} and
address@hidden in the file names:
+
address@hidden
+$ cd bin/myprog
+$ mv TEMPLATE.c myprog.c
+$ mv TEMPLATE.h myprog.h
address@hidden example
+
address@hidden
address@hidden GNU Grep
+Correct all occurances of @code{TEMPLATE} in the input files to
address@hidden (in short or long format). You can get a list of all
+occurances with the following command. If you use Emacs, it will be able to
+parse the Grep output and open the proper file and line automatically. So
+this step can be very easy.
+
address@hidden
+$ grep --color -nHi -e template *
address@hidden example
@item
-Run the following command to re-build the configuration and build system.
+Run the following commands to re-build the configuration and build system,
+and then to configure and build Gnuastro (which now includes your exciting
+new program).
@example
$ autoreconf -f
+$ ./configure
+$ make
@end example
-Your new program will be built the next time you run @command{./configure}
-and @command{make}. You can now start adding your special
-checks/processing.
+
address@hidden
+You are done! You can now start customizing your new program to do your
+special processing. When its complete, just don't forget to add checks
+also, so it can be tested at least once on a user's system with
address@hidden check}, see @ref{Test scripts}. Finally, if you would like to
+share it with all Gnuastro users, inform us so we merge it into Gnuastro's
+main history.
@end enumerate
@@ -26574,10 +27204,6 @@ coordinates.
(@file{astfits}, see @ref{Fits}) View and manipulate FITS file extensions
and header keywords.
address@hidden Statistics
-(@file{aststatistics}, see @ref{Statistics}) Get pixel statistics and
-save histogram and cumulative frequency plots.
-
@item MakeCatalog
(@file{astmkcatalog}, see @ref{MakeCatalog}) Make catalog of labeled image
(output of NoiseChisel). The catalogs are highly customizable and adding
@@ -26598,11 +27224,18 @@ over-sampled image.
that match with each other within a given aperture (may be an ellipse).
@item NoiseChisel
-(@file{astnoisechisel}, see @ref{NoiseChisel}) Detect and segment signal in
-noise. It uses a technique to detect very faint and diffuse, irregularly
-shaped signal in noise (galaxies in the sky), using thresholds that are
-below the Sky value, see @url{http://arxiv.org/abs/1505.01664,
-arXiv:1505.01664}.
+(@file{astnoisechisel}, see @ref{NoiseChisel}) Detect signal in noise. It
+uses a technique to detect very faint and diffuse, irregularly shaped
+signal in noise (galaxies in the sky), using thresholds that are below the
+Sky value, see @url{http://arxiv.org/abs/1505.01664, arXiv:1505.01664}.
+
address@hidden Segment
+(@file{astsegment}, see @ref{Segment}) Segment detected regions based on
+the structure of signal and the input dataset's noise properties.
+
address@hidden Statistics
+(@file{aststatistics}, see @ref{Statistics}) Statistical calculations on
+the input dataset (column in a table, image or datacube).
@item Table
(@file{asttable}, @ref{Table}) Convert FITS binary and ASCII tables into
diff --git a/doc/release-checklist.txt b/doc/release-checklist.txt
index 3cf287f..e6cb40a 100644
--- a/doc/release-checklist.txt
+++ b/doc/release-checklist.txt
@@ -183,6 +183,25 @@ Steps necessary to Package Gnuastro for Debian.
$ sudo apt-get update
$ sudo apt-get upgrade
+ - If this is the first time you are packaging on this system, you will
+ need to install the following programs:
+
+ $ sudo apt-get install devscripts pbuilder pristine-tar git quilt lintian
+ $ sudo pbuilder create
+ $ emacs ~/.devscripts
+
+ Add these two lines to the opened file:
+
+ DEBFULLNAME="Your name"
+ address@hidden
+
+ Then add these lines to `~/.quiltrc':
+
+ QUILT_PATCHES=debian/patches
+ QUILT_NO_DIFF_INDEX=1
+ QUILT_NO_DIFF_TIMESTAMPS=1
+ QUILT_REFRESH_ARGS="-p ab"
+
A restart should help in making sure everything that has been updated is
being used.
diff --git a/lib/Makefile.am b/lib/Makefile.am
index 850c7d7..1d8e6d9 100644
--- a/lib/Makefile.am
+++ b/lib/Makefile.am
@@ -58,9 +58,9 @@ libgnuastro_la_SOURCES = arithmetic.c arithmetic-and.c
arithmetic-bitand.c \
arithmetic-modulo.c arithmetic-multiply.c arithmetic-ne.c \
arithmetic-or.c arithmetic-plus.c array.c binary.c blank.c box.c \
checkset.c convolve.c cosmology.c data.c eps.c fits.c git.c \
- interpolate.c jpeg.c list.c match.c options.c pdf.c permutation.c \
- polygon.c qsort.c dimension.c statistics.c table.c tableintern.c \
- threads.c tiff.c tile.c timing.c txt.c type.c wcs.c
+ interpolate.c jpeg.c label.c list.c match.c options.c pdf.c \
+ permutation.c polygon.c qsort.c dimension.c statistics.c table.c \
+ tableintern.c threads.c tiff.c tile.c timing.c txt.c type.c wcs.c
@@ -70,16 +70,17 @@ libgnuastro_la_SOURCES = arithmetic.c arithmetic-and.c
arithmetic-bitand.c \
# in the $(headersdir) directory. Some of the header files don't need to be
# installed.
headersdir=$(top_srcdir)/lib/gnuastro
-pkginclude_HEADERS = gnuastro/config.h $(headersdir)/arithmetic.h \
- $(headersdir)/array.h $(headersdir)/binary.h $(headersdir)/blank.h \
- $(headersdir)/box.h $(headersdir)/convolve.h $(headersdir)/cosmology.h \
- $(headersdir)/data.h $(headersdir)/dimension.h $(headersdir)/eps.h \
- $(headersdir)/fits.h $(headersdir)/git.h $(headersdir)/jpeg.h \
- $(headersdir)/interpolate.h $(headersdir)/list.h $(headersdir)/match.h \
- $(headersdir)/pdf.h $(headersdir)/permutation.h $(headersdir)/polygon.h \
- $(headersdir)/qsort.h $(headersdir)/statistics.h $(headersdir)/table.h \
- $(headersdir)/threads.h $(headersdir)/tiff.h $(headersdir)/tile.h \
- $(headersdir)/txt.h $(headersdir)/type.h $(headersdir)/wcs.h
+pkginclude_HEADERS = gnuastro/config.h $(headersdir)/arithmetic.h \
+ $(headersdir)/array.h $(headersdir)/binary.h $(headersdir)/blank.h \
+ $(headersdir)/box.h $(headersdir)/convolve.h $(headersdir)/cosmology.h \
+ $(headersdir)/data.h $(headersdir)/dimension.h $(headersdir)/eps.h \
+ $(headersdir)/fits.h $(headersdir)/git.h $(headersdir)/interpolate.h \
+ $(headersdir)/jpeg.h $(headersdir)/label.h $(headersdir)/list.h \
+ $(headersdir)/match.h $(headersdir)/pdf.h $(headersdir)/permutation.h \
+ $(headersdir)/polygon.h $(headersdir)/qsort.h $(headersdir)/statistics.h \
+ $(headersdir)/table.h $(headersdir)/threads.h $(headersdir)/tiff.h \
+ $(headersdir)/tile.h $(headersdir)/txt.h $(headersdir)/type.h \
+ $(headersdir)/wcs.h
@@ -104,8 +105,8 @@ EXTRA_DIST = gnuastro.pc.in $(headersdir)/README
$(internaldir)/README \
$(internaldir)/arithmetic-or.h $(internaldir)/arithmetic-plus.h \
$(internaldir)/checkset.h $(internaldir)/commonopts.h \
$(internaldir)/config.h.in $(internaldir)/fixedstringmacros.h \
- $(internaldir)/options.h $(internaldir)/tableintern.h \
- $(internaldir)/timing.h
+ $(internaldir)/kernel-2d.h $(internaldir)/options.h \
+ $(internaldir)/tableintern.h $(internaldir)/timing.h
diff --git a/lib/data.c b/lib/data.c
index b7eb330..9317b62 100644
--- a/lib/data.c
+++ b/lib/data.c
@@ -533,9 +533,13 @@ gal_data_array_calloc(size_t size)
out[i].type = GAL_TYPE_INVALID;
out[i].ndim = 0;
out[i].dsize = NULL;
+ out[i].size = 0;
+ out[i].mmapname = NULL;
+ out[i].minmapsize = -1;
out[i].nwcs = 0;
out[i].wcs = NULL;
- out[i].mmapname = NULL;
+ out[i].flag = 0;
+ out[i].status = 0;
out[i].next = NULL;
out[i].block = NULL;
out[i].name = out[i].unit = out[i].comment = NULL;
diff --git a/lib/gnuastro-internal/commonopts.h
b/lib/gnuastro-internal/commonopts.h
index 1e41a03..300f498 100644
--- a/lib/gnuastro-internal/commonopts.h
+++ b/lib/gnuastro-internal/commonopts.h
@@ -212,7 +212,7 @@ struct argp_option gal_commonopts_options[] =
GAL_OPTIONS_KEY_OUTPUT,
"STR",
0,
- "Output name.",
+ "Output file name.",
GAL_OPTIONS_GROUP_OUTPUT,
&cp->output,
GAL_TYPE_STRING,
diff --git a/lib/gnuastro-internal/kernel-2d.h
b/lib/gnuastro-internal/kernel-2d.h
new file mode 100644
index 0000000..ccc2d49
--- /dev/null
+++ b/lib/gnuastro-internal/kernel-2d.h
@@ -0,0 +1,129 @@
+/*********************************************************************
+The default 2D kernel to be used in NoiseChisel and Segment.
+This is part of GNU Astronomy Utilities (Gnuastro) package.
+
+Original author:
+ Mohammad Akhlaghi <address@hidden>
+Contributing author(s):
+Copyright (C) 2018, Free Software Foundation, Inc.
+
+Gnuastro is free software: you can redistribute it and/or modify it
+under the terms of the GNU General Public License as published by the
+Free Software Foundation, either version 3 of the License, or (at your
+option) any later version.
+
+Gnuastro is distributed in the hope that it will be useful, but
+WITHOUT ANY WARRANTY; without even the implied warranty of
+MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the GNU
+General Public License for more details.
+
+You should have received a copy of the GNU General Public License
+along with Gnuastro. If not, see <http://www.gnu.org/licenses/>.
+**********************************************************************/
+#ifndef __GAL_KERNEL2D_H__
+#define __GAL_KERNEL2D_H__
+
+
+/* How to produce this kernel
+ ==========================
+
+ Below, the steps necessary to easily create the C contents of this file
+ are described. The first step can be modified to change the default
+ kernel properties and put the new contents into this file.
+
+ Make the kernel
+ ---------------
+
+ We'll first make the kernel using MakeProfiles, then crop the outer
+ layers that are zero. Put these lines into a script and run it.
+
+ set -o errexit # Stop if a program returns false.
+ export GSL_RNG_TYPE=ranlxs2
+ export GSL_RNG_SEED=1
+ astmkprof --kernel=gaussian,2,5 --oversample=1 --envseed -ok2.fits
+ astcrop k2.fits --section=2:*-1,2:*-1 --zeroisnotblank \
+ --mode=img --output=fwhm2.fits
+ rm k2.fits
+
+ Convert it to C code
+ --------------------
+
+ We'll use this tiny C program to convert the FITS file into the two
+ important C variables:
+
+ #include <stdio.h>
+ #include <stdlib.h>
+ #include <gnuastro/fits.h>
+
+ int
+ main(void)
+ {
+ size_t i;
+ float *arr;
+ gal_data_t *img=gal_fits_img_read_to_type("fwhm2.fits", "1",
+ GAL_TYPE_FLOAT32, -1);
+
+ arr=img->array;
+
+ printf("size_t kernel_2d_dsize[2]={%zu, %zu};\n",
+ img->dsize[0], img->dsize[1]);
+ printf("float kernel_2d[%zu]={", img->size);
+ for(i=0;i<img->size;++i)
+ {
+ if(i>0)
+ {
+ if(i % img->dsize[1] == 0 ) printf("\n\n");
+ }
+
+ // We cannot use `\b' here, since we are writing directly
+ // to the command-line, so we'll first write the number,
+ // then decide if any subsequent character (a comma)
+ // should be written.
+ printf("%.7g", arr[i]);
+
+ // The last element doesn't need a comma. In each line,
+ // the last character must not be a space, but for easy
+ // readability, the elements in between need a space.
+ if( i!=(img->size-1) )
+ printf("%s", ((i+1)%img->dsize[1]) ? ", " : ",");
+ }
+ printf("};\n");
+
+ gal_data_free(img);
+ return EXIT_SUCCESS;
+ }
+
+ Run the C program
+ -----------------
+
+ We can now compile and run that C program and put the outputs in
+ `ready.c'. Once its created, copy the contents of `ready.c' after these
+ comments.
+
+ $ astbuildprog -q prepare.c > ready.c
+ */
+
+size_t kernel_2d_dsize[2]={11, 11};
+float kernel_2d[121]={0, 0, 0, 0, 0, 2.570758e-08, 0, 0, 0, 0, 0,
+
+0, 0, 2.981546e-08, 7.249833e-07, 4.468747e-06, 8.409227e-06, 4.554846e-06,
7.034199e-07, 3.002102e-08, 0, 0,
+
+0, 3.054498e-08, 2.614154e-06, 5.891601e-05, 0.0003810036, 0.000708165,
0.0003842406, 5.963722e-05, 2.618934e-06, 2.990584e-08, 0,
+
+0, 7.199899e-07, 5.801019e-05, 0.001365485, 0.009023659, 0.01638159,
0.008892864, 0.001345278, 5.920425e-05, 6.984741e-07, 0,
+
+0, 4.584869e-06, 0.0003830431, 0.008917402, 0.05743425, 0.1061489, 0.05746412,
0.008902563, 0.0003849257, 4.448404e-06, 0,
+
+2.572769e-08, 8.414041e-06, 0.0007008284, 0.0164456, 0.1055995, 0.19753,
0.1061855, 0.01653461, 0.0007141303, 8.41643e-06, 2.550312e-08,
+
+0, 4.582525e-06, 0.0003775396, 0.00898499, 0.05741534, 0.1062144, 0.05700329,
0.008838926, 0.0003822096, 4.543726e-06, 0,
+
+0, 6.883925e-07, 6.09077e-05, 0.001339333, 0.008817007, 0.01636454,
0.008995386, 0.001407854, 6.004799e-05, 7.203602e-07, 0,
+
+0, 3.095966e-08, 2.575403e-06, 5.89859e-05, 0.0003804447, 0.0007091904,
0.0003810006, 5.903253e-05, 2.575202e-06, 2.934356e-08, 0,
+
+0, 0, 3.040937e-08, 7.018197e-07, 4.543086e-06, 8.296753e-06, 4.434901e-06,
6.659026e-07, 3.066215e-08, 0, 0,
+
+0, 0, 0, 0, 0, 2.603901e-08, 0, 0, 0, 0, 0};
+
+#endif
diff --git a/lib/gnuastro/binary.h b/lib/gnuastro/binary.h
index bac8b7c..4ae9357 100644
--- a/lib/gnuastro/binary.h
+++ b/lib/gnuastro/binary.h
@@ -104,4 +104,4 @@ gal_binary_fill_holes(gal_data_t *input, int connectivity,
size_t maxsize);
__END_C_DECLS /* From C++ preparations */
-#endif /* __GAL_DATA_H__ */
+#endif /* __GAL_BINARY_H__ */
diff --git a/lib/gnuastro/dimension.h b/lib/gnuastro/dimension.h
index 2cb434b..90c2c41 100644
--- a/lib/gnuastro/dimension.h
+++ b/lib/gnuastro/dimension.h
@@ -26,6 +26,7 @@ along with Gnuastro. If not, see
<http://www.gnu.org/licenses/>.
/* Include other headers if necessary here. Note that other header files
must be included before the C++ preparations below */
#include <gnuastro/data.h>
+#include <gnuastro/blank.h>
/* C++ Preparations */
#undef __BEGIN_C_DECLS
diff --git a/lib/gnuastro/label.h b/lib/gnuastro/label.h
new file mode 100644
index 0000000..e73b7d8
--- /dev/null
+++ b/lib/gnuastro/label.h
@@ -0,0 +1,74 @@
+/*********************************************************************
+label -- Work on labeled (positive integer valued) datasets.
+This is part of GNU Astronomy Utilities (Gnuastro) package.
+
+Original author:
+ Mohammad Akhlaghi <address@hidden>
+Contributing author(s):
+Copyright (C) 2018, Free Software Foundation, Inc.
+
+Gnuastro is free software: you can redistribute it and/or modify it
+under the terms of the GNU General Public License as published by the
+Free Software Foundation, either version 3 of the License, or (at your
+option) any later version.
+
+Gnuastro is distributed in the hope that it will be useful, but
+WITHOUT ANY WARRANTY; without even the implied warranty of
+MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the GNU
+General Public License for more details.
+
+You should have received a copy of the GNU General Public License
+along with Gnuastro. If not, see <http://www.gnu.org/licenses/>.
+**********************************************************************/
+#ifndef __GAL_LABEL_H__
+#define __GAL_LABEL_H__
+
+/* Include other headers if necessary here. Note that other header files
+ must be included before the C++ preparations below */
+#include <gnuastro/data.h>
+
+/* C++ Preparations */
+#undef __BEGIN_C_DECLS
+#undef __END_C_DECLS
+#ifdef __cplusplus
+# define __BEGIN_C_DECLS extern "C" {
+# define __END_C_DECLS }
+#else
+# define __BEGIN_C_DECLS /* empty */
+# define __END_C_DECLS /* empty */
+#endif
+/* End of C++ preparations */
+
+
+
+/* Actual header contants (the above were for the Pre-processor). */
+__BEGIN_C_DECLS /* From C++ preparations */
+
+
+
+
+
+/* Constants for the clump over-segmentation. */
+#define GAL_LABEL_INIT -1
+#define GAL_LABEL_RIVER -2
+#define GAL_LABEL_TMPCHECK -3
+
+
+
+
+
+/* Functions. */
+size_t
+gal_label_oversegment(gal_data_t *input, gal_data_t *indexs,
+ gal_data_t *label, size_t *topinds);
+
+void
+gal_label_grow_indexs(gal_data_t *labels, gal_data_t *indexs, int withrivers,
+ int connectivity);
+
+
+
+
+__END_C_DECLS /* From C++ preparations */
+
+#endif /* __GAL_LABEL_H__ */
diff --git a/lib/label.c b/lib/label.c
new file mode 100644
index 0000000..538f704
--- /dev/null
+++ b/lib/label.c
@@ -0,0 +1,529 @@
+/*********************************************************************
+label -- Work on labeled (integer valued) datasets.
+This is part of GNU Astronomy Utilities (Gnuastro) package.
+
+Original author:
+ Mohammad Akhlaghi <address@hidden>
+Contributing author(s):
+Copyright (C) 2018, Free Software Foundation, Inc.
+
+Gnuastro is free software: you can redistribute it and/or modify it
+under the terms of the GNU General Public License as published by the
+Free Software Foundation, either version 3 of the License, or (at your
+option) any later version.
+
+Gnuastro is distributed in the hope that it will be useful, but
+WITHOUT ANY WARRANTY; without even the implied warranty of
+MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the GNU
+General Public License for more details.
+
+You should have received a copy of the GNU General Public License
+along with Gnuastro. If not, see <http://www.gnu.org/licenses/>.
+**********************************************************************/
+#include <config.h>
+
+#include <stdio.h>
+#include <errno.h>
+#include <error.h>
+#include <stdlib.h>
+
+#include <gnuastro/list.h>
+#include <gnuastro/qsort.h>
+#include <gnuastro/label.h>
+#include <gnuastro/dimension.h>
+
+
+
+
+
+
+/****************************************************************
+ ***************** Over segmentation ********************
+ ****************************************************************/
+/* Over-segment the region specified by its indexs into peaks and their
+ respective regions (clumps). This is very similar to the immersion
+ method of Vincent & Soille(1991), but here, we will not separate the
+ image into layers, instead, we will work based on the ordered flux
+ values. If a certain pixel (at a certain level) has no neighbors, it is
+ a local maximum and will be assigned a new label. If it has a labeled
+ neighbor, it will take that label and if there is more than one
+ neighboring labeled region that pixel will be a `river` pixel.
+
+ DON'T FORGET: SET THE FLAGS FOR CONV EQUAL TO INPUT IN SEGMENT.
+
+*/
+size_t
+gal_label_oversegment(gal_data_t *input, gal_data_t *indexs,
+ gal_data_t *label, size_t *topinds)
+{
+ size_t ndim=input->ndim;
+
+ float *arr=input->array;
+ gal_list_sizet_t *Q=NULL, *cleanup=NULL;
+ size_t *a, *af, ind, *dsize=input->dsize;
+ size_t *dinc=gal_dimension_increment(ndim, dsize);
+ int32_t n1, nlab, rlab, curlab=1, *clabel=label->array;
+
+ /*********************************************
+ For checks and debugging:*
+ gal_data_t *crop;
+ size_t extcount=1;
+ int32_t *cr, *crf;
+ size_t checkdsize[2]={10,10};
+ size_t checkstart[2]={50,145};
+ char *filename="clumpbuild.fits";
+ size_t checkstartind=gal_dimension_coord_to_index(2, dsize, checkstart);
+ gal_data_t *tile=gal_data_alloc(gal_data_ptr_increment(arr, checkstartind,
+ input->type),
+ GAL_TYPE_INVALID, 2, checkdsize,
+ NULL, 0, 0, NULL, NULL, NULL);
+ tile->block=input;
+ gal_checkset_writable_remove(filename, 0, 0);
+ if(p->cp.numthreads!=1)
+ error(EXIT_FAILURE, 0, "in the debugging mode of `clumps_oversegment' "
+ "only one thread must be used");
+ crop=gal_data_copy(tile);
+ gal_fits_img_write(crop, filename, NULL, PROGRAM_NAME);
+ gal_data_free(crop);
+ printf("blank: %u\nriver: %u\ntmpcheck: %u\ninit: %u\n",
+ (int32_t)GAL_BLANK_INT32, (int32_t)GAL_LABEL_RIVER,
+ (int32_t)GAL_LABEL_TMPCHECK, (int32_t)GAL_LABEL_INIT);
+ tile->array=gal_tile_block_relative_to_other(tile, clabel);
+ tile->block=clabel;
+ **********************************************/
+
+
+ /* If the size of the indexs is zero, then this function is pointless. */
+ if(indexs->size==0) return 0;
+
+
+ /* Sort the given indexs based on their flux (`gal_qsort_index_arr' is
+ defined as static in `gnuastro/qsort.h') */
+ gal_qsort_index_arr=input->array;
+ qsort(indexs->array, indexs->size, sizeof(size_t),
+ gal_qsort_index_float_decreasing);
+
+
+ /* Initialize the region we want to over-segment. */
+ af=(a=indexs->array)+indexs->size;
+ do clabel[*a]=GAL_LABEL_INIT; while(++a<af);
+
+
+ /* Go over all the given indexs and pull out the clumps. */
+ af=(a=indexs->array)+indexs->size;
+ do
+ /* When regions of a constant flux or masked regions exist, some later
+ indexs (although they have same flux) will be filled before hand. If
+ they are done, there is no need to do them again. */
+ if(clabel[*a]==GAL_LABEL_INIT)
+ {
+ /* It might happen where one or multiple regions of the pixels
+ under study have the same flux. So two equal valued pixels of
+ two separate (but equal flux) regions will fall immediately
+ after each other in the sorted list of indexs and we have to
+ account for this.
+
+ Therefore, if we see that the next pixel in the index list has
+ the same flux as this one, it does not guarantee that it should
+ be given the same label. Similar to the breadth first search
+ algorithm for finding connected components, we will search all
+ the neighbours and the neighbours of those neighbours that have
+ the same flux of this pixel to see if they touch any label or
+ not and to finally give them all the same label. */
+ if( (a+1)<af && arr[*a]==arr[*(a+1)] )
+ {
+ /* Label of first neighbor found. */
+ n1=0;
+
+ /* A small sanity check. */
+ if(Q!=NULL || cleanup!=NULL)
+ error(EXIT_FAILURE, 0, "%s: a bug! Please contact us at %s so "
+ "we can fix this problem. `Q' and `cleanup' should be "
+ "NULL but while checking the equal flux regions they "
+ "aren't", __func__, PACKAGE_BUGREPORT);
+
+ /* Add this pixel to a queue. */
+ gal_list_sizet_add(&Q, *a);
+ gal_list_sizet_add(&cleanup, *a);
+ clabel[*a] = GAL_LABEL_TMPCHECK;
+
+ /* Find all the pixels that have the same flux and are
+ connected. */
+ while(Q!=NULL)
+ {
+ /* Pop an element from the queue. */
+ ind=gal_list_sizet_pop(&Q);
+
+ /* Look at the neighbors and see if we already have a
+ label. */
+ GAL_DIMENSION_NEIGHBOR_OP(ind, ndim, dsize, ndim, dinc,
+ {
+ /* If it is already decided to be a river, then stop
+ looking at the neighbors. */
+ if(n1!=GAL_LABEL_RIVER)
+ {
+ /* For easy reading. */
+ nlab=clabel[ nind ];
+
+ /* This neighbor's label isn't zero. */
+ if(nlab)
+ {
+ /* If this neighbor has not been labeled yet
+ and has an equal flux, add it to the queue
+ to expand the studied region.*/
+ if( nlab==GAL_LABEL_INIT && arr[nind]==arr[*a] )
+ {
+ clabel[nind]=GAL_LABEL_TMPCHECK;
+ gal_list_sizet_add(&Q, nind);
+ gal_list_sizet_add(&cleanup, nind);
+ }
+ else
+ n1=( nlab>0
+
+ /* If this neighbor has a positive
+ nlab, it belongs to another object,
+ so if `n1' has not been set for the
+ whole region (n1==0), put `nlab'
+ into `n1'. If `n1' has been set and
+ is different from `nlab' then this
+ whole equal flux region should be a
+ wide river because it is connecting
+ two connected regions.*/
+ ? ( n1
+ ? (n1==nlab ? n1 : GAL_LABEL_RIVER)
+ : nlab )
+
+ /* If the data has blank pixels (recall
+ that blank in int32 is negative),
+ see if the neighbor is blank and if
+ so, set the label to a river. Since
+ the flag checking can be done
+ outside this loop, for datasets with
+ no blank element this last step will
+ be completley ignored. */
+ : ( ( (input->flag
+ & GAL_DATA_FLAG_HASBLANK)
+ && nlab==GAL_BLANK_INT32 )
+ ? GAL_LABEL_RIVER : n1 ) );
+ }
+
+ /* If this neigbour has a label of zero, then we
+ are on the edge of the indexed region (the
+ neighbor is not in the initial list of pixels
+ to segment). When over-segmenting the noise and
+ the detections, `clabel' is zero for the parts
+ of the image that we are not interested in
+ here. */
+ else clabel[*a]=GAL_LABEL_RIVER;
+ }
+ } );
+ }
+
+ /* Set the label that is to be given to this equal flux
+ region. If `n1' was set to any value, then that label should
+ be used for the whole region. Otherwise, this is a new
+ label, see the case for a non-flat region. */
+ if(n1) rlab = n1;
+ else
+ {
+ rlab = curlab++;
+ if( topinds ) /* This is a local maximum of */
+ topinds[rlab]=*a; /* this region, save its index. */
+ }
+
+ /* Give the same label to the whole connected equal flux
+ region, except those that might have been on the side of
+ the image and were a river pixel. */
+ while(cleanup!=NULL)
+ {
+ ind=gal_list_sizet_pop(&cleanup);
+ /* If it was on the sides of the image, it has been
+ changed to a river pixel. */
+ if( clabel[ ind ]==GAL_LABEL_TMPCHECK ) clabel[ ind ]=rlab;
+ }
+ }
+
+ /* The flux of this pixel is not the same as the next sorted
+ flux, so simply find the label for this object. */
+ else
+ {
+ /* `n1' is the label of the first labeled neighbor found, so
+ we'll initialize it to zero. */
+ n1=0;
+
+ /* Go over all the fully connected neighbors of this pixel and
+ see if all the neighbors (with maximum connectivity: the
+ number of dimensions) that have a non-macro value belong to
+ one label or not. If the pixel is neighboured by more than
+ one label, set it as a river pixel. Also if it is touching a
+ zero valued pixel (which does not belong to this object),
+ set it as a river pixel.*/
+ GAL_DIMENSION_NEIGHBOR_OP(*a, ndim, dsize, ndim, dinc,
+ {
+ /* When `n1' has already been set as a river, there is no
+ point in looking at the other neighbors. */
+ if(n1!=GAL_LABEL_RIVER)
+ {
+ /* For easy reading. */
+ nlab=clabel[ nind ];
+
+ /* If this neighbor is on a non-processing label, then
+ set the first neighbor accordingly. Note that we
+ also want the zero valued neighbors (detections if
+ working on sky, and sky if working on detection):
+ we want rivers between the two domains. */
+ n1 = ( nlab
+
+ /* nlab is non-zero. */
+ ? ( nlab>0
+
+ /* Neighbor has a meaningful label, so
+ check with any previously found labeled
+ neighbors. */
+ ? ( n1
+ ? ( nlab==n1 ? n1 : GAL_LABEL_RIVER )
+ : nlab )
+
+ /* If the dataset has blank values and this
+ neighbor is blank, then the pixel should
+ be a river. Note that the blank checking
+ can be optimized out, so if the input
+ doesn't have blank values,
+ `nlab==GAL_BLANK_INT32' will never be
+ checked. */
+ : ( (input->flag & GAL_DATA_FLAG_HASBLANK)
+ && nlab==GAL_BLANK_INT32
+ ? GAL_LABEL_RIVER : n1 ) )
+
+ /* `nlab==0' (the neighbor lies in the other
+ domain (sky or detections). To avoid the
+ different domains touching, this pixel
+ should be a river. */
+ : GAL_LABEL_RIVER );
+ }
+ });
+
+ /* Either assign a new label to this pixel, or give it the one
+ of its neighbors. If n1 equals zero, then this is a new
+ peak, and a new label should be created. But if n1!=0, it
+ is either a river pixel (has more than one labeled neighbor
+ and has been set to `GAL_LABEL_RIVER' before) or all its
+ neighbors have the same label. In both such cases, rlab
+ should be set to n1.*/
+ if(n1) rlab = n1;
+ else
+ {
+ rlab = curlab++;
+ if( topinds )
+ topinds[ rlab ]=*a;
+ }
+
+ /* Put the found label in the pixel. */
+ clabel[ *a ] = rlab;
+ }
+
+ /*********************************************
+ For checks and debugging:
+ if( *a / dsize[1] >= checkstart[0]
+ && *a / dsize[1] < checkstart[0] + checkdsize[0]
+ && *a % dsize[1] >= checkstart[1]
+ && *a % dsize[1] < checkstart[1] + checkdsize[1] )
+ {
+ printf("%zu (%zu: %zu, %zu): %u\n", ++extcount, *a,
+ (*a%dsize[1])-checkstart[1], (*a/dsize[1])-checkstart[0],
+ clabel[*a]);
+ crop=gal_data_copy(tile);
+ crf=(cr=crop->array)+crop->size;
+ do if(*cr==GAL_LABEL_RIVER) *cr=0; while(++cr<crf);
+ gal_fits_img_write(crop, filename, NULL, PROGRAM_NAME);
+ gal_data_free(crop);
+ }
+ **********************************************/
+ }
+ while(++a<af);
+
+ /*********************************************
+ For checks and debugging:
+ tile->array=NULL;
+ gal_data_free(tile);
+ printf("Total number of clumps: %u\n", curlab-1);
+ **********************************************/
+
+ /* Clean up. */
+ free(dinc);
+
+ /* Return the total number of clumps. */
+ return curlab-1;
+}
+
+
+
+
+
+/* Grow the given labels over the list of indexs. In over-segmentation
+ (watershed algorithm), the indexs have to be sorted by pixel value and
+ local maximums (that aren't touching any already labeled pixel) get a
+ separate label. However, here the final number of labels will not
+ change. All pixels that aren't directly touching a labeled pixel just
+ get pushed back to the start of the loop, and the loop iterates until
+ its size doesn't change any more. This is because in a generic scenario
+ some of the indexed pixels might not be reachable. As a result, it is
+ not necessary for the given indexs to be sorted here.
+
+ This function looks for positive-valued neighbors and will label a pixel
+ if it touches one. Therefore, it is is very important that only
+ pixels/labels that are intended for growth have positive values before
+ calling this function. Any non-positive (zero or negative) value will be
+ ignored by this function. Thus, it is recommended that while filling in
+ the `indexs' array values, you initialize all those pixels with
+ `GAL_LABEL_INIT', and set non-labeled pixels that you don't want to grow
+ to 0.
+
+ This function will write into both the input datasets. After this
+ function, some of the non-positive `labels' pixels will have a new
+ positive label and the number of useful elements in `indexs' will have
+ decreased. Those indexs that couldn't be labeled will remain inside
+ `indexs'. If `withrivers' is non-zero, then pixels that are immediately
+ touching more than one positive value will be given a `GAL_LABEL_RIVER'
+ label.
+
+ Note that the `indexs->array' is not re-allocated to its new size at the
+ end. But since `indexs->dsize[0]' and `indexs->size' have new values,
+ the extra elements just won't be used until they are ultimately freed by
+ `gal_data_free'.
+
+ Input:
+
+ labels: The labels array that must be operated on. The pixels that
+ must be "grown" must have the value `GAL_LABEL_INIT' (negative).
+
+ sorted_indexs: The indexs of the pixels that must be grown.
+
+ withrivers: as described above.
+
+ connectivity: connectivity to define neighbors for growth. */
+void
+gal_label_grow_indexs(gal_data_t *labels, gal_data_t *indexs, int withrivers,
+ int connectivity)
+{
+ int searchngb;
+ size_t *iarray=indexs->array;
+ int32_t n1, nlab, *olabel=labels->array;
+ size_t *s, *sf, thisround, ninds=indexs->size;
+ size_t *dinc=gal_dimension_increment(labels->ndim, labels->dsize);
+
+ /* Some basic sanity checks: */
+ if(labels->type!=GAL_TYPE_INT32)
+ error(EXIT_FAILURE, 0, "%s: `labels' has to have type of int32_t",
+ __func__);
+ if(indexs->type!=GAL_TYPE_SIZE_T)
+ error(EXIT_FAILURE, 0, "%s: `indexs' must be `size_t' type but it is "
+ "`%s'", __func__, gal_type_name(indexs->type,1));
+ if(indexs->ndim!=1)
+ error(EXIT_FAILURE, 0, "%s: `indexs' has to be a 1D array, but it is "
+ "%zuD", __func__, indexs->ndim);
+
+ /* The basic idea is this: after growing, not all the blank pixels are
+ necessarily filled, for example the pixels might belong to two regions
+ above the growth threshold. So the pixels in between them (which are
+ below the threshold will not ever be able to get a label). Therefore,
+ the safest way we can terminate the loop of growing the objects is to
+ stop it when the number of pixels left to fill in this round
+ (thisround) equals the number of blanks.
+
+ To start the loop, we set `thisround' to one more than the number of
+ indexed pixels. Note that it will be corrected immediately after the
+ loop has started, it is just important to pass the `while'. */
+ thisround=ninds+1;
+ while( thisround > ninds )
+ {
+ /* `thisround' will keep the number of pixels to be inspected in this
+ round. `ninds' will count the number of pixels left without a
+ label by the end of this round. Since `ninds' comes from the
+ previous loop (or outside, for the first round) it has to be saved
+ in `thisround' to begin counting a fresh. */
+ thisround=ninds;
+ ninds=0;
+
+ /* Go over all the available indexs. NOTE: while the `indexs->array'
+ pointer remains unchanged, `indexs->size' can/will change (get
+ smaller) in every loop. */
+ sf = (s=indexs->array) + indexs->size;
+ do
+ {
+ /* We'll begin by assuming the nearest neighbor of this pixel
+ has no label (has a value of 0). */
+ n1=0;
+
+ /* Check the neighbors of this pixel. Note that since this
+ macro has multiple loops within it, we can't use
+ break. We'll use the `searchngb' variable instead. */
+ searchngb=1;
+ GAL_DIMENSION_NEIGHBOR_OP(*s, labels->ndim, labels->dsize,
+ connectivity, dinc,
+ {
+ if(searchngb)
+ {
+ /* For easy reading. */
+ nlab = olabel[nind];
+
+ /* This neighbor's label is meaningful. */
+ if(nlab>0) /* This is a real label. */
+ {
+ if(n1) /* A prev. ngb label has been found. */
+ {
+ if( n1 != nlab ) /* Different label from */
+ { /* prevously found ngb for this pixel. */
+ n1=GAL_LABEL_RIVER;
+ searchngb=0;
+ }
+ }
+ else
+ { /* This is the first labeld neighbor found. */
+ n1=nlab;
+
+ /* If we want to completely fill in the region
+ (`withrivers==0'), then there is no point in
+ looking in other neighbors, the first
+ neighbor we find, is the one we'll use. */
+ if(!withrivers) searchngb=0;
+ }
+ }
+ }
+ } );
+
+ /* The loop over neighbors (above) finishes with three
+ possibilities:
+
+ n1==0 --> No labeled neighbor was found.
+ n1==GAL_LABEL_RIVER --> Connecting two labeled regions.
+ n1>0 --> Only has one neighbouring label.
+
+ The first one means that no neighbors were found and this
+ pixel should be kept for the next loop (we'll be growing the
+ objects pixel-layer by pixel-layer). In the other two cases,
+ we just need to write in the value of `n1'. */
+ if(n1)
+ {
+ /* Set the label. */
+ olabel[*s]=n1;
+
+ /* If this pixel is a river (can only happen when
+ `withrivers' is zero), keep it in the loop, because we
+ want the `indexs' dataset to contain all non-positive
+ (non-labeled) pixels, including rivers. */
+ if(n1==GAL_LABEL_RIVER)
+ iarray[ ninds++ ] = *s;
+ }
+ else
+ iarray[ ninds++ ] = *s;
+
+ /* Correct the size of the `indexs' dataset. */
+ indexs->size = indexs->dsize[0] = ninds;
+ }
+ while(++s<sf);
+ }
+
+ /* Clean up. */
+ free(dinc);
+}
diff --git a/tests/Makefile.am b/tests/Makefile.am
index 9032a48..31e4664 100644
--- a/tests/Makefile.am
+++ b/tests/Makefile.am
@@ -124,9 +124,11 @@ if COND_MATCH
match/merged-cols.sh: prepconf.sh.log
endif
if COND_MKCATALOG
- MAYBE_MKCATALOG_TESTS = mkcatalog/simple.sh mkcatalog/aperturephot.sh
+ MAYBE_MKCATALOG_TESTS = mkcatalog/detections.sh \
+ mkcatalog/objects-clumps.sh mkcatalog/aperturephot.sh
- mkcatalog/simple.sh: noisechisel/noisechisel.sh.log
+ mkcatalog/objects-clumps.sh: segment/segment.sh.log
+ mkcatalog/detections.sh: noisechisel/noisechisel.sh.log
mkcatalog/aperturephot.sh: noisechisel/noisechisel.sh.log \
mkprof/clearcanvas.sh.log
endif
@@ -153,6 +155,11 @@ if COND_NOISECHISEL
noisechisel/noisechisel.sh: mknoise/addnoise.sh.log
endif
+if COND_SEGMENT
+ MAYBE_SEGMENT_TESTS = segment/segment.sh
+
+ segment/segment.sh: noisechisel/noisechisel.sh.log
+endif
if COND_STATISTICS
MAYBE_STATISTICS_TESTS = statistics/basicstats.sh statistics/estimate_sky.sh
@@ -220,13 +227,14 @@ lib/multithread.sh: mkprof/mosaic1.sh.log
# Final Tests
# ===========
-TESTS = prepconf.sh lib/multithread.sh lib/versioncxx.sh \
+TESTS = prepconf.sh lib/multithread.sh lib/versioncxx.sh \
$(MAYBE_ARITHMETIC_TESTS) $(MAYBE_BUILDPROG_TESTS) \
$(MAYBE_CONVERTT_TESTS) $(MAYBE_CONVOLVE_TESTS) $(MAYBE_COSMICCAL_TESTS) \
$(MAYBE_CROP_TESTS) $(MAYBE_FITS_TESTS) $(MAYBE_MATCH_TESTS) \
$(MAYBE_MKCATALOG_TESTS) $(MAYBE_MKNOISE_TESTS) $(MAYBE_MKPROF_TESTS) \
- $(MAYBE_NOISECHISEL_TESTS) $(MAYBE_STATISTICS_TESTS) \
- $(MAYBE_SUBTRACTSKY_TESTS) $(MAYBE_TABLE_TESTS) $(MAYBE_WARP_TESTS)
+ $(MAYBE_NOISECHISEL_TESTS) $(MAYBE_SEGMENT_TESTS) \
+ $(MAYBE_STATISTICS_TESTS) $(MAYBE_SUBTRACTSKY_TESTS) \
+ $(MAYBE_TABLE_TESTS) $(MAYBE_WARP_TESTS)
diff --git a/tests/arithmetic/snimage.sh b/tests/arithmetic/snimage.sh
index 18f719a..a42d95e 100755
--- a/tests/arithmetic/snimage.sh
+++ b/tests/arithmetic/snimage.sh
@@ -24,7 +24,8 @@
# file exists (basicchecks.sh is in the source tree).
prog=arithmetic
execname=../bin/$prog/ast$prog
-img=convolve_spatial_noised_labeled.fits
+imgin=convolve_spatial_noised.fits
+imgnc=convolve_spatial_noised_labeled.fits
@@ -41,7 +42,8 @@ img=convolve_spatial_noised_labeled.fits
# - The input data was not made (for example the test that created the
# data file failed).
if [ ! -f $execname ]; then echo "$execname not created."; exit 77; fi
-if [ ! -f $img ]; then echo "$img does not exist."; exit 77; fi
+if [ ! -f $imgin ]; then echo "$imgin does not exist."; exit 77; fi
+if [ ! -f $imgnc ]; then echo "$imgnc does not exist."; exit 77; fi
@@ -49,5 +51,5 @@ if [ ! -f $img ]; then echo "$img does not exist.";
exit 77; fi
# Actual test script
# ==================
-$execname $img $img - $img / --hdu=1 --hdu=4 --hdu=5 \
+$execname $imgin $imgnc - $imgnc / --hdu=1 --hdu=SKY --hdu=SKY_STD \
--output=snimage.fits
diff --git a/tests/mkcatalog/aperturephot.sh b/tests/mkcatalog/aperturephot.sh
index 196049b..31466e7 100755
--- a/tests/mkcatalog/aperturephot.sh
+++ b/tests/mkcatalog/aperturephot.sh
@@ -51,6 +51,6 @@ if [ ! -f $objimg ]; then echo "$objimg does not exist";
exit 77; fi
# Actual test script
# ==================
-$execname $img --objectsfile=$objimg --objectshdu=1 \
- --output=aperturephot.fits \
+$execname $objimg --hdu=1 --valuesfile=$img \
+ --output=aperturephot.fits \
--objid --x --y --ra --dec --magnitude --sn
diff --git a/tests/mkcatalog/simple.sh b/tests/mkcatalog/detections.sh
similarity index 96%
copy from tests/mkcatalog/simple.sh
copy to tests/mkcatalog/detections.sh
index 71a3ebb..d47c94a 100755
--- a/tests/mkcatalog/simple.sh
+++ b/tests/mkcatalog/detections.sh
@@ -50,4 +50,4 @@ if [ ! -f $img ]; then echo "$img does not exist.";
exit 77; fi
# Actual test script
# ==================
$execname $img --x --y --ra --dec --magnitude --upperlimitmag --sn \
- --tableformat=txt
+ --tableformat=txt --hdu=DETECTIONS
diff --git a/tests/mkcatalog/simple.sh b/tests/mkcatalog/objects-clumps.sh
similarity index 91%
rename from tests/mkcatalog/simple.sh
rename to tests/mkcatalog/objects-clumps.sh
index 71a3ebb..4fa0b46 100755
--- a/tests/mkcatalog/simple.sh
+++ b/tests/mkcatalog/objects-clumps.sh
@@ -24,7 +24,7 @@
# file exists (basicchecks.sh is in the source tree).
prog=mkcatalog
execname=../bin/$prog/ast$prog
-img=convolve_spatial_noised_labeled.fits
+img=convolve_spatial_noised_labeled_segmented.fits
@@ -50,4 +50,4 @@ if [ ! -f $img ]; then echo "$img does not exist.";
exit 77; fi
# Actual test script
# ==================
$execname $img --x --y --ra --dec --magnitude --upperlimitmag --sn \
- --tableformat=txt
+ --tableformat=txt --clumpscat --output=objects-clumps.txt
diff --git a/tests/noisechisel/noisechisel.sh b/tests/noisechisel/noisechisel.sh
index ef9ebdf..d2bf1b2 100755
--- a/tests/noisechisel/noisechisel.sh
+++ b/tests/noisechisel/noisechisel.sh
@@ -1,4 +1,4 @@
-# Detect objects and clumps in an image using NoiseChisel.
+# Detect objects in an image using NoiseChisel.
#
# See the Tests subsection of the manual for a complete explanation
# (in the Installing gnuastro section).
@@ -41,7 +41,7 @@ img=convolve_spatial_noised.fits
# - The input data was not made (for example the test that created the
# data file failed).
if [ ! -f $execname ]; then echo "$execname not created."; exit 77; fi
-if [ ! -f $img ]; then echo "$img does not exist."; exit 77; fi
+if [ ! -f $img ]; then echo "$img does not exist."; exit 77; fi
@@ -49,5 +49,5 @@ if [ ! -f $img ]; then echo "$img does not exist.";
exit 77; fi
# Actual test script
# ==================
-$execname $img --cleangrowndet --checkdetection --checksegmentation \
- --continueaftercheck --tilesize=100,100
+$execname $img --tilesize=100,100 --snquant=0.999 --cleangrowndet \
+ --checkdetection --continueaftercheck
diff --git a/tests/prepconf.sh b/tests/prepconf.sh
index 0035607..d82d8de 100755
--- a/tests/prepconf.sh
+++ b/tests/prepconf.sh
@@ -72,8 +72,9 @@ rm addedoptions.txt
# easy readability. Note that some programs may need to build their
# configuration files during compilation. Hence, their configuration files
# are in the build directory, not the source directory.
-for prog in arithmetic buildprog convertt convolve cosmiccal crop fits \
- match mkcatalog mknoise mkprof noisechisel statistics table warp
+for prog in arithmetic buildprog convertt convolve cosmiccal crop fits \
+ match mkcatalog mknoise mkprof noisechisel segment \
+ statistics table warp
do
if test -f $topsrc/bin/$prog/ast$prog.conf; then
ctopdir=$topsrc
diff --git a/tests/arithmetic/snimage.sh b/tests/segment/segment.sh
similarity index 89%
copy from tests/arithmetic/snimage.sh
copy to tests/segment/segment.sh
index 18f719a..6b9c038 100755
--- a/tests/arithmetic/snimage.sh
+++ b/tests/segment/segment.sh
@@ -1,4 +1,4 @@
-# Make an S/N image to test Arithmetic.
+# Segment the detections into clumps and objects.
#
# See the Tests subsection of the manual for a complete explanation
# (in the Installing gnuastro section).
@@ -22,7 +22,7 @@
# Set the variables (The executable is in the build tree). Do the
# basic checks to see if the executable is made or if the defaults
# file exists (basicchecks.sh is in the source tree).
-prog=arithmetic
+prog=segment
execname=../bin/$prog/ast$prog
img=convolve_spatial_noised_labeled.fits
@@ -49,5 +49,4 @@ if [ ! -f $img ]; then echo "$img does not exist.";
exit 77; fi
# Actual test script
# ==================
-$execname $img $img - $img / --hdu=1 --hdu=4 --hdu=5 \
- --output=snimage.fits
+$execname $img --tilesize=100,100 --snquant=0.99
| [Prev in Thread] |
Current Thread |
[Next in Thread] |
- [gnuastro-commits] master 5a2f382: New Segment, NoiseChisel only detection, MakeCatalog re-write,
Mohammad Akhlaghi <=