Re: columnview dynamic block - different time summing behaviour for EFFORT and CLOCKSUM

[Alexander Adolf]

Ihor Radchenko <yantar92@posteo.net> writes:

> [...]
> Calling `org-columns--clean-item' is a must to create a valid table.

True.

Additionally, it would seem advisable to call `org-quote-vert' on the
data, too, as `org-columns--clean-item' does not take care of vertical
bars? This is done in a previous step in `org-columns--capture-view',
however, so that the vertical bars get converted to "\vert" before the
formatting function gets called.

`org-link-heading-search-string', and `org-link-make-string' (both
called from the formatting function _after_ `org-columns--clean-item')
OTOH take care of the link's path and description parts being
appropriate for a link.

Thus, we have both layers of safeguarding: "link-safe" data is generated
from "table-safe" data.

> [...]
>> 2) Considering what happens when I do `org-store-link' and
>>    `org-insert-link', it would seem more advisable to:
>> [...]
>
> I do not recall seeing `org-store-link' in the patches you submitted.
> So, I am not sure what you are talking about. May you elaborate?
> [...]

Apologies for not having been clear enough. With "when I do" I wanted to
refer to the "cleanup" of headline text when using `org-store-link' and
`org-insert-link' interactively.

I have moved the link generation to the formatting function as
mentioned. This also removes the need for any new function parameters,
since the formatting function receives the dynamic block's parameter
list, which contains a :link property when set in the #+BEGIN line.

Kindly find updated patches below. I hope to have succeeded in
addressing all your comments; that was my intention at least.


Many thanks and looking forward to your thoughts,

  --alexander


>From a36dba46a63d7d879bf2faf3c4c885b9ea3e28ad Mon Sep 17 00:00:00 2001
From: Alexander Adolf <alexander.adolf@condition-alpha.com>
Date: Sun, 14 Apr 2024 18:14:05 +0200
Subject: [PATCH 1/2] lisp/org-colview.el: Add formatter parameter to colview
 dynamic block

* lisp/org-colview.el (org-dblock-write:column view): Factor out the
existing formatting code to new function
`org-columns-dblock-write-default', and honour new dblock parameter
:formatter for specifying a different formatting function.
(org-columns-dblock-write-default): New function with current
formatting code.
(org-columns--capture-view): Amend docstring to better explain the
format of the data being passed to the formatting function.
(org-clock-clocktable-formatter): New option to define a global
default formatting function, defaulting to the current behaviour.
* doc/org-manual.org (Capturing column view): Describe new :formatter
parameter.
* etc/ORG-NEWS (New option ~org-columns-dblock-formatter~): Announce
new option.
(=colview= dynamic block supports custom formatting function):
Describe new custom formatting function feature.
---
 doc/org-manual.org  |  7 +++++++
 etc/ORG-NEWS        | 29 +++++++++++++++++++++++++++++
 lisp/org-colview.el | 41 +++++++++++++++++++++++++++++++++++------
 3 files changed, 71 insertions(+), 6 deletions(-)

diff --git a/doc/org-manual.org b/doc/org-manual.org
index d24230568..000783772 100644
--- a/doc/org-manual.org
+++ b/doc/org-manual.org
@@ -6014,6 +6014,13 @@ This dynamic block has the following parameters:
   Specify a column attribute (see [[*Column attributes]]) for the dynamic
   block.
 
+- =:formatter= ::
+
+  #+cindex: @samp{formatter}, dynamic block parameter
+  #+vindex: org-columns-dblock-formatter
+  A function to format column view data and insert it into the buffer.
+  See the option ~org-columns-dblock-formatter~.
+
 The following commands insert or update the dynamic block:
 
 - ~org-columns-insert-dblock~ ::
diff --git a/etc/ORG-NEWS b/etc/ORG-NEWS
index dbf849422..84ea7cde9 100644
--- a/etc/ORG-NEWS
+++ b/etc/ORG-NEWS
@@ -999,6 +999,16 @@ even though it does not have its own ID.  By giving files 
top-level id
 properties, links to headlines in the file can also be made more
 robust by using the file id instead of the file path.
 
+*** New option ~org-columns-dblock-formatter~
+
+=colview= dynamic blocks now understand a new ~:formatter~ parameter
+to use a specific function for formatting and inserting the contents
+of the dynamic block. This new option can be used to set the global
+default formatting function that will be used for =colview= dynamic
+blocks that do not specify any ~:formatter~ parameter. Its default
+value (the new function ~org-columns-dblock-write-default~) yields the
+previous (fixed) formatting behaviour.
+
 ** New features
 *** ~org-paste-subtree~ now handles =C-u= and =C-u C-u= prefix arguments 
specially
 
@@ -1035,6 +1045,25 @@ Example:
 : | PROYECTO EMACS |
 : #+END:
 
+*** =colview= dynamic block supports custom formatting function
+
+The =colview= dynamic block understands a new ~:formatter~ parameter,
+which specifies a user-supplied function to format and insert the data
+in the dynamic block.
+
+A global default formatting function for =colview= dynamic blocks can
+be set via the new option ~org-columns-dblock-formatter~ which
+defaults to the new function ~org-columns-dblock-write-default~, that
+implements the previous (fixed) formatting behaviour. Hence, the
+default behaviour is identical to previous versions.
+
+The global default function can be overridden for any given =colview=
+dynamic block individually by specifying a custom formatter function
+using the new ~:formatter~ parameter on the block's =BEGIN= line.
+
+This new feature replicates the ~:formatter~ option already available
+for =clocktable= dynamic blocks.
+
 *** =ob-tangle.el=: New flag to remove tangle targets before writing
 
 When ~org-babel-tangle-remove-file-before-write~ is set to ~t~ the
diff --git a/lisp/org-colview.el b/lisp/org-colview.el
index 4718d1db7..7a6471377 100644
--- a/lisp/org-colview.el
+++ b/lisp/org-colview.el
@@ -123,6 +123,12 @@ in `org-columns-summary-types-default', which see."
                              (function :tag "Summarize")
                              (function :tag "Collect")))))
 
+(defcustom org-columns-dblock-formatter #'org-columns-dblock-write-default
+  "Function to format data in column view dynamic blocks.
+For more information, see `org-columns-dblock-write-default'."
+  :group 'org-properties
+  :package-version '(Org . "9.7")
+  :type 'function)
 
 
 ;;; Column View
@@ -1442,9 +1448,13 @@ that will be excluded from the resulting view.  FORMAT 
is a
 format string for columns, or nil.  When LOCAL is non-nil, only
 capture headings in current subtree.
 
-This function returns a list containing the title row and all
-other rows.  Each row is a list of fields, as strings, or
-`hline'."
+This function returns a list containing the title row and all other
+rows.  Each row is either a list, or the symbol `hline'.  The first list
+is the heading row as a list of strings with the column titles according
+to FORMAT.  All subsequent lists each represent a body row as a list
+whose first element is an integer indicating the outline level of the
+entry, and whose remaining elements are strings with the contents for
+the columns according to FORMAT."
   (org-columns (not local) format)
   (goto-char org-columns-top-level-marker)
   (let ((columns (length org-columns-current-fmt-compiled))
@@ -1545,8 +1555,15 @@ PARAMS is a property list of parameters:
 `:vlines'
 
     When non-nil, make each column a column group to enforce
-    vertical lines."
-  (let ((table
+    vertical lines.
+
+`:formatter'
+
+    A function to format the data and insert it into the
+    buffer.  Overrides the default formatting function set in
+    `org-columns-dblock-formatter'."
+  (let ((insert-pos (point))
+        (table
         (let ((id (plist-get params :id))
               view-file view-pos)
           (pcase id
@@ -1573,7 +1590,17 @@ PARAMS is a property list of parameters:
                                         (plist-get params :exclude-tags)
                                         (plist-get params :format)
                                         view-pos)))))
-        (width-specs
+        (formatter (or (plist-get params :formatter)
+                       org-columns-dblock-formatter
+                       #'org-columns-dblock-write-default)))
+    (funcall formatter insert-pos table params)))
+
+(defun org-columns-dblock-write-default (ipos table params)
+  "Write out a columnview table at position IPOS in the current buffer.
+TABLE is a table with data as produced by `org-columns--capture-view'.
+PARAMS is the parameter property list obtained from the dynamic block
+definition."
+  (let ((width-specs
          (mapcar (lambda (spec) (nth 2 spec))
                  org-columns-current-fmt-compiled)))
     (when table
@@ -1616,6 +1643,8 @@ PARAMS is a property list of parameters:
         ;; to the resulting table, adding alignment field as the first
         ;; row.
         (push (mapcar (lambda (width) (when width (format "<%d>" width))) 
width-specs) table))
+      ;; now insert the table into the buffer
+      (goto-char ipos)
       (let ((content-lines (org-split-string (plist-get params :content) "\n"))
            recalc)
        ;; Insert affiliated keywords before the table.
-- 
2.39.3 (Apple Git-146)

>From 6c3fd68419ac2be705f29bd56dcd23b01b6da86b Mon Sep 17 00:00:00 2001
From: Alexander Adolf <alexander.adolf@condition-alpha.com>
Date: Mon, 15 Apr 2024 18:01:40 +0200
Subject: [PATCH 2/2] lisp/org-colview.el: Add link parameter to colview
 dynamic block

* lisp/org-colview.el (org-columns--capture-view): Add new link
parameter, which when non-nil causes ITEM headlines to be linked to
their origins.
(org-dblock-write:columnview): Pass new link parameter to
`org-columns--capture-view', and explain its use in the docstring.
* doc/org-manual.org (Capturing column view): Describe new :link
parameter.
* etc/ORG-NEWS (=colview= dynamic block can link to headlines):
Describe new link feature.
---
 doc/org-manual.org  |  5 +++++
 etc/ORG-NEWS        |  6 ++++++
 lisp/org-colview.el | 33 +++++++++++++++++++++++----------
 3 files changed, 34 insertions(+), 10 deletions(-)

diff --git a/doc/org-manual.org b/doc/org-manual.org
index 000783772..a77492971 100644
--- a/doc/org-manual.org
+++ b/doc/org-manual.org
@@ -6009,6 +6009,11 @@ This dynamic block has the following parameters:
 
   When non-~nil~, indent each =ITEM= field according to its level.
 
+- =:link= ::
+
+  When non-~nil~, link the =ITEM= headlines in the table to their
+  origins.
+
 - =:format= ::
 
   Specify a column attribute (see [[*Column attributes]]) for the dynamic
diff --git a/etc/ORG-NEWS b/etc/ORG-NEWS
index 84ea7cde9..b4aea10b8 100644
--- a/etc/ORG-NEWS
+++ b/etc/ORG-NEWS
@@ -1064,6 +1064,12 @@ using the new ~:formatter~ parameter on the block's 
=BEGIN= line.
 This new feature replicates the ~:formatter~ option already available
 for =clocktable= dynamic blocks.
 
+*** =colview= dynamic block can link to headlines
+
+The =colview= dynamic block understands a new ~:link~ parameter, which
+when non-~nil~ causes =ITEM= headlines in the table to be linked to
+their origins.
+
 *** =ob-tangle.el=: New flag to remove tangle targets before writing
 
 When ~org-babel-tangle-remove-file-before-write~ is set to ~t~ the
diff --git a/lisp/org-colview.el b/lisp/org-colview.el
index 7a6471377..74f7a8f58 100644
--- a/lisp/org-colview.el
+++ b/lisp/org-colview.el
@@ -1441,12 +1441,12 @@ and variances (respectively) of the individual 
estimates."
 (defun org-columns--capture-view (maxlevel match skip-empty exclude-tags 
format local)
   "Get the column view of the current buffer.
 
-MAXLEVEL sets the level limit.  SKIP-EMPTY tells whether to skip
-empty rows, an empty row being one where all the column view
-specifiers but ITEM are empty.  EXCLUDE-TAGS is a list of tags
-that will be excluded from the resulting view.  FORMAT is a
-format string for columns, or nil.  When LOCAL is non-nil, only
-capture headings in current subtree.
+MAXLEVEL sets the level limit.  SKIP-EMPTY tells whether to skip empty
+rows, an empty row being one where all the column view specifiers but
+ITEM are empty.  EXCLUDE-TAGS is a list of tags that will be excluded
+from the resulting view.  FORMAT is a format string for columns, or nil.
+When LOCAL is non-nil, only capture headings in current subtree.  When
+LINK is non-nil, item headlines will be linked to their origins.
 
 This function returns a list containing the title row and all other
 rows.  Each row is either a list, or the symbol `hline'.  The first list
@@ -1557,6 +1557,10 @@ PARAMS is a property list of parameters:
     When non-nil, make each column a column group to enforce
     vertical lines.
 
+`:link'
+
+    Link the item headlines in the table to their origins.
+
 `:formatter'
 
     A function to format the data and insert it into the
@@ -1600,9 +1604,10 @@ PARAMS is a property list of parameters:
 TABLE is a table with data as produced by `org-columns--capture-view'.
 PARAMS is the parameter property list obtained from the dynamic block
 definition."
-  (let ((width-specs
-         (mapcar (lambda (spec) (nth 2 spec))
-                 org-columns-current-fmt-compiled)))
+  (let ((link (plist-get params :link))
+       (width-specs
+        (mapcar (lambda (spec) (nth 2 spec))
+                org-columns-current-fmt-compiled)))
     (when table
       ;; Prune level information from the table.  Also normalize
       ;; headings: remove stars, add indentation entities, if
@@ -1626,7 +1631,15 @@ definition."
                           (and (numberp hlines) (<= level hlines))))
              (push 'hline new-table))
            (when item-index
-             (let ((item (org-columns--clean-item (nth item-index (cdr row)))))
+             (let* ((raw (nth item-index (cdr row)))
+                    (cleaned (org-columns--clean-item raw))
+                    (item (if link
+                              (let ((search (org-link-heading-search-string 
raw)))
+                                (org-link-make-string
+                                 (if (not (buffer-file-name)) search
+                                   (format "file:%s::%s" (buffer-file-name) 
search))
+                                 cleaned))
+                            cleaned)))
                (setf (nth item-index (cdr row))
                      (if (and indent (> level 1))
                          (concat "\\_" (make-string (* 2 (1- level)) ?\s) item)
-- 
2.39.3 (Apple Git-146)