Re: bug#47885: [PATCH] org-table-import: Make it more smarter for interactive use

[Maxim Nikulin]

On 17/05/2021 12:29, Bastien wrote:
> Utkarsh Singh <utkarsh190601@gmail.com> writes:
> > For now can you review the patches I proposed earlier in this
> > thread?
>
> Not until both you and Maxim are confident this is useful, complete
> and predictable.

I have too many points to object to consider my opinion as objective.

I am unsure if colon as a separator (passwd "db") is widely used case. I 
was surprised that it is impossible to implement locale-aware detection 
of semicolon as separator due to limitations of Emacs that is not 
friendly in respect to internationalization of number formatting.

Bastien, I do not know how much tests you prefer to have in Org. Nicolas 
asked for extensive tests 
https://orgmode.org/list/875z07jx6n.fsf@nicolasgoaziou.fr I think, 
before starting work on tests, it is necessary to decide if the patches 
are acceptable in general.

Personally, I have realized that I would prefer to have anything related 
to CSV (besides very basic features) in a dedicated package, e.g. in 
csv-mode. It might define a special yank handler for copy-paste to 
org-mode. Unsure if the author of csv-mode agrees with my point of view. 
Another option is to pass files through python code to take advantage of 
more advanced heuristics.

On 15/05/2021 18:09, https://orgmode.org/list/87im3kdzi5.fsf@gmail.com 
Utkarsh Singh wrote:
> --- a/lisp/org-table.el
> +++ b/lisp/org-table.el
> @@ -954,7 +954,8 @@ lines.  It can have the following values:
>  - (64)    Prompt for a regular expression as field separator.
>  - integer When a number, use that many spaces, or a TAB, as field separator.
>  - regexp  When a regular expression, use it to match the separator."
> -  (interactive "f\nP")
> +  (interactive (list (read-file-name "Import file: ")
> +                     (prefix-numeric-value current-prefix-arg)))
>    (when (and (called-interactively-p 'any)

Sending patches, I am afraid to break something I was not aware about.

I have read docstrings for the modified functions and tried to import 
the following file "tbl.csv" with some CSV features. LibreOffice imports 
it correctly, it even normalizes 66.3e-35 to 6.63e-34 (I can not say 
that I always appreciate such silent modifications).

1,Word,66.3e-35
2,Unquoted cell,2.7
3,"Quoted cell",3.14
4,"Cell ""with quotes""",2021-06-01
5,"Next cell is empty",""
6,"Cell with new
Line",6.28

My optimistic expectation was (OK, I did not believe I got such result):

| 1 | Word               |   66.3e-35 |
| 2 | Unquoted cell      |        2.7 |
| 3 | Quoted cell        |       3.14 |
| 4 | Cell "with quotes" | 2021-06-01 |
| 5 | Next cell is empty |            |
| 6 | Cell with new      |       6.28 |
|   | Line               |            |

Org 9.1.9 M-x org-table-import
and C-u M-x org-table-import
actual results are close enough to my expectations

|     1 | Word               |   66.3e-35 |
|     2 | Unquoted cell      |        2.7 |
|     3 | Quoted cell        |       3.14 |
|     4 | Cell "with quotes" | 2021-06-01 |
|     5 | Next cell is empty |            |
|     6 | "Cell with new     |            |
| Line" | 6.28               |            |

M-x org-table-import RET tbl RET
completes file name to tbl.csv

Org 9.4.5+patches M-x org-table-import

| 1,Word,66.3e-35 |            |                      |           |
| 2,Unquoted      | cell,2.7   |                      |           |
| 3,"Quoted       | cell",3.14 |                      |           |
| 4,"Cell         | ""with     | quotes""",2021-06-01 |           |
| 5,"Next         | cell       | is                   | empty","" |
| 6,"Cell         | with       | new                  |           |
| Line",6.28      |            |                      |           |

Org 9.4.5+patches C-u M-x org-table-import

| 1,Word,66.3e-35                     |
| 2,Unquoted cell,2.7                 |
| 3,"Quoted cell",3.14                |
| 4,"Cell ""with quotes""",2021-06-01 |
| 5,"Next cell is empty",""           |
| 6,"Cell with new                    |
| Line",6.28                          |

M-x org-table-import RET tbl RET
complains that file name extension is not txt, tsv or csv.

So my personal conclusion is that CSV file is imported incorrectly in 
both cases: with guessed separator and with explicitly requested through 
prefix argument. Completion works a bit worse too.

One more note concerning locale support.

On 18/05/2021 22:05, Utkarsh Singh wrote:
> On 2021-05-18, 19:31 +0700, Maxim Nikulin wrote:
> > The question may be risen in emacs-devel but I am unsure if I will 
> > participate in discussion.
>
> Why?

I am aware of some problems related to localization but I do not have 
consistent vision what API emacs should have. I have no idea what 
information is available in Windows. That is why I expect that 
discussion may be time consuming while I am not sure that someone will 
be ready to implement new features.