Re: [O] org-mode and python pandas

[Eric Schulte]

Dov Grobgeld <address@hidden> writes:

> Thanks for the answers, but there is still something missing in order
> to get it to work. Part of it seems to be connected to the python
> parsing. E.g. the following translation of Eric's sh example doesn't
> output correctly with python:
>
>
> #+BEGIN_SRC python :results output
> print """,A,B,C
> 0,0.628365,0.424279,0.619791
> 1,0.799666,0.527572,0.132928
> 2,0.837255,0.138906,0.408233
> 3,0.388080,0.146212,0.575346
> """
> #+END_SRC
>
> #+RESULTS:
> : ,A,B,C
> : 0,0.628365,0.424279,0.619791
> : 1,0.799666,0.527572,0.132928
> : 2,0.837255,0.138906,0.408233
> : 3,0.388080,0.146212,0.575346
>
>
> #+BEGIN_SRC python :results table
> return """,A,B,C
> 0,0.628365,0.424279,0.619791
> 1,0.799666,0.527572,0.132928
> 2,0.837255,0.138906,0.408233
> 3,0.388080,0.146212,0.575346
> """
> #+END_SRC
>
> #+RESULTS:
> | 
> ,A,B,C\n\n0,0.628365,0.424279,0.619791\n\n1,0.799666,0.527572,0.132928\n\n2,0.837255,0.138906,0.408233\n\n3,0.388080,0.146212,0.575346
> |
>
> It seems that the only way to get a table from python is by outputting
> a two dimensional python structure:
>
> #+BEGIN_SRC python
> return [[0,0.628365,0.424279,0.619791],
>         [1,0.799666,0.527572,0.132928]]
> #+END_SRC
>
> #+RESULTS:
> | 0 | 0.628365 | 0.424279 | 0.619791 |
> | 1 | 0.799666 | 0.527572 | 0.132928 |
>
> This seems quite limiting....
>

In most cases this is what one wants when returning data from python
code.  The following elisp defined a "panda" code block, which is just
like python, only it assumes that the results will be these sort of
human readable strings instead of python code.

    ;; -*- emacs-lisp -*-
    (defun org-babel-execute:panda (body params)
      (let ((results
             (org-babel-execute:python
              body (org-babel-merge-params '((:results . "scalar")) params))))
        (org-babel-result-cond (cdr (assoc :result-params params))
          results
          (let ((tmp-file (org-babel-temp-file "sh-")))
            (with-temp-file tmp-file (insert results))
            (org-babel-import-elisp-from-file tmp-file)))))

With the above evaluated the following works

    #+BEGIN_SRC panda
    return """,A,B,C
    0,0.628365,0.424279,0.619791
    1,0.799666,0.527572,0.132928
    2,0.837255,0.138906,0.408233
    3,0.388080,0.146212,0.575346
    """
    #+END_SRC

    #+RESULTS:
    |   |        A |        B |        C |
    | 0 | 0.628365 | 0.424279 | 0.619791 |
    | 1 | 0.799666 | 0.527572 | 0.132928 |
    | 2 | 0.837255 | 0.138906 | 0.408233 |
    | 3 |  0.38808 | 0.146212 | 0.575346 |

>
> Another related question is if there is any support for header tables?
> I.e. instead of this:
>
> |   |        A |        B |        C |
> | 0 | 0.827817 | 0.664009 | 0.089161 |
> | 1 | 0.170031 | 0.729214 | 0.110918 |
> | 2 | 0.575918 | 0.863924 | 0.757536 |
> | 3 | 0.682722 | 0.774445 | 0.992041 |
>
> I want this:
>
> |   |        A |        B |        C |
> |---+----------+----------+----------|
> | 0 | 0.827817 | 0.664009 | 0.089161 |
> | 1 | 0.170031 | 0.729214 | 0.110918 |
> | 2 | 0.575918 | 0.863924 | 0.757536 |
> | 3 | 0.682722 | 0.774445 | 0.992041 |
>
> I guess that if I start playing around with the python ob module, it
> should be possible to get this working?
>

See the :colnames header argument in the manual.

Best,

>
> Regards,
> Dov
>
> On Mon, Jul 1, 2013 at 8:04 PM, Rasmus <address@hidden> wrote:
>> Achim Gratz <address@hidden> writes:
>>
>>>>> 2. Add to pandas the option of globally influencing the text
>>>>> formatting so that it outputs something more parsable by org-mode.
>>>>
>>>> This sounds promising, if pandas support csv output that will be
>>>> correctly parsed by Org-mode.
>>>
>>> The package already has CSV export, so one could use that.  I don't know
>>> if you could echo the result directly to the output, all examples
>>> revolve around putting the CSV into a file.  For Org, TSV output would
>>> be more natural.
>>
>> Something like:
>>
>> from pandas import DataFrame
>> from numpy.random import rand
>> from sys import stdout
>> df = DataFrame(rand(10,3), columns = list('abc'))
>> df
>> df.to_csv(stdout, sep="\t", header = True, cols=(1,2))
>>
>> I was completely unable to get ob-python working this morning, so I
>> haven't tested it.  I'm using python3, build in python mode and elpy.
>>
>> In any case, the csv route might be better, as Pandas doesn't print
>> the table if it's too big (try changing 10 to 1000 above).
>>
>> --
>> Powered by magic pixies!
>>
>>
>

-- 
Eric Schulte
http://cs.unm.edu/~eschulte