Re: One vs many directories

[Jean Louis]

* Ihor Radchenko <yantar92@gmail.com> [2020-11-23 08:43]:
> >> I am wondering what you mean by Org's philosophy. Why would it have 
> >> anything to do with directories?
> >
> > Org's philosophy is to have one or a handful of directories without
> > nesting of directories.  Users are not expected to have their Org
> > files in a deeply nested tree.  Org also prefers big files with large
> > trees rather than lots of little files.
> >
> > By philosophy, I mean the dev consensus on the correct way to do
> > things, and coded configuration and usability biases.
> 
> I believe that org support all possibilities. The user can decide to
> keep many (possibly nested) org files, a few large org files, or
> anywhere in between. There are several parallel feature sets allowing to
> work in a single file as well as with a bunch of smaller files.

Yes, sure, and I guess you mentioned some people have problems with
many files. And I have no problem at all with many files as they are
per subject separated, per person or per subject separated. They are
not hyperlinked to each other, it is me who make system to hyperlink
to files.

Searching for Joe Doe, F4 and I am in Org file for Joe Doe. My
personal TODO list need not really show the tasks assigned to Joe Doe,
I could show only * TODO Joe Doe and when I click there then I can get
all tasks for Joe Doe as new Org file.

It means I am accessing hundreds of Org files from the meta level by
using conceptual location backed by the database.

Some people maybe access multiple Org files through Agenda, me I
don't. Some items are "non existent" and I do not know how to ask
agenda to refresh itself. This is not big deal as I do not access
items throgh Agenda, though I find it very useful. 

org-agenda is trying to put all tasks and notes from various files
into one list and that is of course not so easy task considering that
files can be anywhere on the file system and that they need to be
"remembered". 

> For a single file, the user can search headings with org-goto (without a
> need to explicitly travel through all the nesting headline levels),
> reveal only headings satisfying certain keyword/tag/any other search
> criteria with org-sparse-tree, or built agenda views restricted to a
> single file (or even subtree).

M-x org-goto is useful feature to find headlines. And I never use it,
just standard Emacs search is enough within a file. Meanings I am
searching are often inside of the headline. And it is not my perosnal
way locating things.

Personally, I am using parts of Org, like specific headling to export
it and to send to remote person, or to print the file as project and
to bind it nicely.

When I see repetitive action, for example that I have to send "Daily
Report" template to a person by email, than I just bookmark that in
Emacs with {C-x r m} under something that I think is the meaning of
it. Then I forget about it. Next time when I need to send report, I am
{C-x r b} and quickly completing it and then I am exporting and
inserting into the email.

In general I speak of subsets or sub-lists among lists. List of Org
files is one list, list of headings within Org file is other list, and
list of specific subject related headings or bookmarks to such is
third subset of lists.

> For multiple files located anywhere in the filesystem, there is always
> org-refile capable of filing the information to proper place
> searching deeply nested headlines with ease regardless of the file the
> information is physically located in. Headlines from multiple files can
> be grouped using agenda views for any given search criteria (showing
> todo items or items for a single day/week is just a tiny subset of what
> agenda can do).

That may be useful for those who find it while my use case is
different, here is how it is for me:

** TODO Heading [1/2] [50%]

1) [X] Do this
2) [ ] Do that

that is my personal use case. I do not do things like:

** TODO Do this

Description of the task

And so far I know org-refile works on headings. It does not work on
list items.

Sometimes the task describes something that belongs to other file, I
just kill and yank to other file. And I keep RCS revision control
system of files.

As user may have many various sparse tasks to do or notes that require
action and attention in soonest future it is best to consolidate tasks
into one centralized system.

Such system should encompass all tasks or notes that require attention
or action in soonest future and should offer constant reminders to
user on what has to be done and when and which people are related to
the task.

When I mentioned "sparse tasks" I refer to my usage and handling of
mess:

1) Bunch of Org files, org-agenda and Org mode tries to accommodate me
   by consolidating everything into lists

2) There are hundreds of such tasks all over, Org tries to consolidate
   it.

3) There are various tasks and actions to do that are not recorded in
   Org files, those cannot be handled by Org.

4) There are database based Tasks in several groupings, some are just
   tagged with TODO, some are recorded in actual action requiring
   groups.

Instead of attempting to be perfect by using Org files for me
personally is best to consolidate all tasks in the database as such
are related either to people mostly or to some subjects related to me
personally which again belongs to "People".

And Org file for people need not have a task there, it can be exported
automatically into the Org.

- when searching for Person Robert S., I locate person and press F4

- Org file for that person is automatically created

- heading * Tasks is automatically created and expanded there by using SQL

Concept is here:

* Tasks

#+BEGIN_SRC sql :engine postgresql :exports results :results value raw 
SELECT '** [[(todo ' || simpletodos_id || ')][TODO]] ' || 
simpletodos_datecreated::date || ' ' ||
simpletodos_name || E'\n\n' || simpletodos_description FROM simpletodos
WHERE simpletodos_contacts = 23187;
#+END_SRC

#+RESULTS:
** [[(todo120)][TODO]] 2017-11-07 Do something

Something I need to do. (THIS IS HEADING OR TASK DESCRIPTION).

The SQL query need to defined only one time and not in the Org file,
but in the program that creates the Org file for the user. Instead of
the SQL query in the specific Org file it could be a simple Emacs
expression that never changes such as (tasks-for-user)

Then the SQL query generates the headings under #+RESULTS: and those
headings come from centralized consolidated database of tasks.

It is then interesting that this column of the database can include
any kind of the mode that Emacs supports, it could include any kind of
file, be it image, video or anything as long as such task is assigned
to the user. It could include other Org files and collection of Org
files or just description or database based whole Org file if
necessary. It becomes very abstract and liberated from limits.

That way I would be editing tasks on the meta level by clicking on
TODO link. If task would be done and completed it could be shown in
separate section of Org file related to user.

Additional buttons can be automatically included such as:

- All tasks for user -> Hyperlink to meta level consolidated TODO management

- Add task for user

- Re-assign from this user to other user

That is using Org mode as viewer of various information, not only as
handler of the information.

The cruft with org-agenda is then removed as then by pressing F5 I get
the list of all tasks and I could filter it how I wish. Which is
similar to org-agenda. The list is coming from the database and is
blazing fast.

Then I can filter using helm or ivy completion or built-in completion:

- I can simply type name of person related to the task, be it myself
  or somebody else. Majority of "my" tasks are related to other
  people. I search by people, sometimes by companies or organizations.

- I can type subject name or tag, any tag that need not be defined in
  single Org file and I find the task

- then if task is related to the person, I could quickly jump into
  persons meta report, from where there is Org files, other files,
  picture links all summarized in the meta Org profile, similar like
  FBI profiles we see on movies, one find the person's name and can
  see anything about that person.

- or I could delete the task, re-assign, modify the task. It becomes
  semi-automatically modified in the Org file of the user.

Can I automated the execution of Babel code upon opening of the Org
file?

When all tasks become centralized by any means, in my case in the
central database, then Org files get liberated from tasks and become
structural and relational. I can edit each task and I can always see
the updated list of the tasks and relations from one object to the
other. Hyperlinks get automatically created.

Personal problem is that tasks are sparse and separate in various Org
files and not centralized. I become dependent of org-agenda to do what
I need but it never does what I need.

Example is the need to relate task to people. Tasks are people related.

Purchasing cloths for your daughter? People.

Visiting museum? People related.

Some people are in Kenya, I do not think always on them. But I might
when I come there. Then I would write "Kenya" to get people I need to
put attention on. If tasks are centralized I can quickly jump to list
of Kenyan people.

If tasks are not centralized I need to put some tag under all those
people's files "Kenya" which is repetitive and error prone activity.

And I get trapped into "Org mode" for years. Which I am now, I am
trapped but it does not help me to manage things how I think.

I do value org-agenda but it is large effort to make SQL query out of
the Org mode. And very expensive effort as it is huge program:

-rw-r--r-- 1 415K Oct 19 10:20 org-agenda.el

If I organize all my tasks centrally and only display them in Org
files related to people, then making similar features like org-agenda
becomes trivial.

- Agenda for current week becomes trivial SQL query such as to select
  those tasks by current week. Result becomes a list that may be used
  either in Emacs completion or tabulated list mode or in Org or any
  kind of modes.

- To list entries with any kind of tags also become trivial, at least
  for centralized database tasks that have been tagged.

- searching for keywords is trivial single line SQL query.

- multi-occur is not trivial, as that would involve searching in all
  Org files. It asks for indexing Org files and going over them.

- Finding any FLAGGED entries would become trivial

It opens plethora of various means of reporting and accessing various
action related notes or tasks.

By general principle and due to nature of tasks that they do need
attention, I think that all tasks should be centralized, any how, it
does not matter how. From that principle I find it better that
majority of tasks are handled in just few files and not many files as
it becomes complex and users cannot any more remember which files.

org-agenda and Org try to remember that for user, but there are many
ways how Org files can be left without attention.

If my staff member in other country uses Emacs such can access
database and see assigned tasks. I can also export centralized tasks
for the user and send such to user as Org file.

If user updates tasks, as long as hyperlinks in the user's Org file
are not disturbed, I can review the work of the user and just click on
the hyperlink to update such task or mark them as DONE. Opinion of
staff member on what is done and what is not done is not necessarily
my opinion.

Description of content body of the heading can be programmatically
captured and entered into centralized database as a task.

Then we comes to actual execution of tasks. How do we get reminded?

Is the reminder only if I press {C-c a} for org-agenda? Do I need to
do action to get reminded?

I think computer shall be programmed to remind me on the start. Tasks
that are in the database can be shown directly in Emacs, but SQL
queries can be run and shown upon first log in into computer. Emails
could be sent, SMS could be sent to remind me or other people assigned
to the task. This brings attention and where people put attention that
is becoming reality.

Tasks could be shown by using timer in the mini buffer to remind user
of what has to be done and what is next, what is today to be done.

Keys can be assigned to TODAY, WEEK, MONTH, YEAR.

Birthdays can be consolidated automatically, as if there is database
of people, then their birtdays are automatic types of notes or tasks.

People who travel often would like to see other people in the city. By
changing one's location one could get list of known people and places
in the city. If I am in Mombasa, Kenya I may forget that good
restaurant as there are not many and visitor has to be brought in good
one, not bad one (which there are many).

With the introduction of emacs-libpq into GNU ELPA such features may
become reality.

emacs-libpq @ Github
https://github.com/anse1/emacs-libpq

Summary:

- tasks are people related

- plethora of Org files makes hard life to developers trying to satisfie users' 
needs

- tasks should be centralized and related to objects such as people, 
organizations, subjects

- Org files may be used as viewers for centralized task systems with
  all the features left as they are as all properties and tags and
  anything can be obtained from the database. Nothing changes.

- upon saving Org file or upon click or invocation, the task in the
  database can be updated if some description has been changed, but
  updates can take place in the database directly.

- tasks get liberated from any limits and formats, they can be just
  hyperlinks or hyperdocuments to just anything.

- integration and relation to many other objects becomes easier possible

- searching, indexing, making plans, or creating new meta Org files on
  the fly by using SQL queries becomes trivial. org-agenda of 450k can
  become easily redundant
  
- once centralized, there must be reminding system that works within
  or without Emacs, tasks are accessed by blazing speed, related
  people's files are quickly located, various queries and ways of
  reporting tasks in the list or meta Org file on the fly becomes
  possible.

I use "meta Org file" only to say that format is useful for displaying
structured and relational information for the reason that it has
hyperlinking support. As long as hyperlinks can be shown in any mode
then reports could be displayed any how.