[Orgmode] Re: Literate Programming with Org mode

[Sébastien Vauban]

Hi Eric,

"Eric Schulte" wrote:
> Sébastien Vauban <address@hidden> writes:
>> "Eric Schulte" wrote:
>>> "Eric Schulte" <address@hidden> writes:
>
> Thanks for giving this a try, comments below...

You're welcome. I'm more than interested into this. And I play the easy role.
Just testing... and commenting ;-)

Sorry for answering late, though.


>> [...] the code should be included via the LaTeX `listings' package (thus,
>> colored contextually to the programming language) and not as verbatim.
>
> I agree fontified code would certainly be preferable to the current approach
> of inserting source-code as verbatim. This seems to be a general org-mode
> development issue rather than org-babel specific, so I'm happy to push the
> buck on this one :). I don't know much about the listings package, but would
> another possibly be easier to use htmlize.el (used for fontification on html
> export) along with html -> latex conversion?

I've tried the following new settings on a freshly pulled Org-mode (this
morning):

--8<---------------cut here---------------start------------->8---
;; tell org to use listings
(setq org-export-latex-listings t)

;; you must include the listings package
(add-to-list 'org-export-latex-packages-alist '("" "listings"))

;; if you want colored source code, then you need to include the
;; (x)color package
(add-to-list 'org-export-latex-packages-alist '("" "xcolor")))
--8<---------------cut here---------------end--------------->8---

though, with no luck, right now.


> Please do another git pull before trying the steps below so we can be sure
> we're working off the same code base.

Done one hour ago.


>>> Calling `org-babel-tangle' from within this file will create the
>>> Payment.sql, Lessons.sql, and Enterprise.sql source-code files.
>>
>> Nope...
>>
>> Open ~/Personal/Templates/org-babel-lit-prog.org
>> tangled 0 source-code blocks
>
> I have changed the original org-babel-lit-prog.org so that it should work
> with your configuration of odd levels only and posted it up here [1].
>
>   Note that you can grab all of these referenced files by cloning the git
>   repo at git://github.com/eschulte/noweb-example.git.

Using your repo.


> By calling org-babel-tangle from within this org-mode file I was able to
> generate Enterprise.sql [2], Lessons.sql [3], and Payment.sql [4].

I still couldn't. Void variable: file.

Now (I've done some changes, explained below), I can...

--8<---------------cut here---------------start------------->8---
Stored: Header
Stored: Condition to be satisfied
Stored: File Enterprise
Stored: File Lessons
Stored: File Payment
Added to 
/home/sva/Downloads/emacs/site-lisp/noweb-example/w-babel/Enterprise.sql
Added to /home/sva/Downloads/emacs/site-lisp/noweb-example/w-babel/Lessons.sql
Added to /home/sva/Downloads/emacs/site-lisp/noweb-example/w-babel/Payment.sql
tangled 3 source-code blocks
--8<---------------cut here---------------end--------------->8---

Great!

Though, this is not entirely what I expected. Let's take the Enterprise.sql
file:

--8<---------------cut here---------------start------------->8---
-- generated by org-babel-tangle

-- 
[[file:~/Downloads/emacs/site-lisp/noweb-example/w-babel/org-babel-lit-prog.org::*File%20Enterprise][block-3]]
-- <<sql-init>>
SELECT abcID, etpID, etpAssurATPolNum
FROM enterprise JOIN record
    ON (etpAbcID_fk = abcID)
WHERE etpAbcID_fk
-- <<sql-cond>>
-- block-3 ends here
--8<---------------cut here---------------end--------------->8---

I'm expecting the sql-init and sql-cond blocks to be replaced by their defined
equivalent. So, in this case, I should have the following file:

--8<---------------cut here---------------start------------->8---
-- generated by org-babel-tangle

-- 
[[file:~/Downloads/emacs/site-lisp/noweb-example/w-babel/org-babel-lit-prog.org::*File%20Enterprise][block-3]]
-- Hey, Emacs! This is a -*- coding: utf-8 -*- file!

-- no longer display the count message
SET NOCOUNT ON

DECLARE @dateFmtStyleIn int; SET @dateFmtStyleIn = 120 -- ODBC canonical
DECLARE @dateFmtStyleOut int; SET @dateFmtStyleOut = 103 -- French dd/mm/yyyy

DECLARE @firstDayOfThisMonth smalldatetime
SET @firstDayOfThisMonth = CONVERT(smalldatetime,
                                   CAST(YEAR(GETDATE()) AS char(4)) + '-'
                                   + CAST(MONTH(GETDATE()) AS char(2)) + '-'
                                   + '01' + ' 00:00:00',
                                   @dateFmtStyleIn)

SELECT abcID, etpID, etpAssurATPolNum
FROM enterprise JOIN record
    ON (etpAbcID_fk = abcID)
WHERE etpAbcID_fk

      IN (SELECT actAbcID_fk
          FROM status
          WHERE (staID = 338 AND staEtat = 3))
      AND abcSignDate < @firstDayOfThisMonth
  ORDER BY abcID
-- block-3 ends here
--8<---------------cut here---------------end--------------->8---

Do we agree on this?


>> Is it normal that you don't include the extension in the tangle property?
>> Does it add the language extension automatically?
>
> Yes, org-babel adds the language specific extension automatically.  This
> could be changed/overridden if it proves to limiting.  It grew out of
> the fact that originally source-code files were named based upon the
> basename of the org-mode file and the language extension.

I find that this is OK right now. Though, people could want to give an
explicit extension, moreover for `.htm' or `.html' files where standard
conventions are not that clear.


>>> The document can also be exported into html and latex using org-mode's
>>> standard export functionality.
>>
>> I have problems as well...
>
> Again maybe this is the result of conflicting values of org-odd-levels-only?
> Please try this with the newer org-mode file [1]. I was able to export this
> to latex resulting in the following tex file [5] and pdf file [6]. I was
> also able to export this file to html which includes code highlighting [7].
>
>> Thanks for all! This is seriously a hot topic you've almost solved...
>
> Sorry I can't be more specific with answers to your problems tangling and
> exporting. Please do give the new org-mode file [1] a try with the latest
> version org-babel and let me know if the problems persist.

I still had the same problem in the beginning: around the `verbatim'
environments, I had wrong `\LaTeX{}' environments created...

Though, I found the problem: it's somehow a conflict with the
`org-special-blocks' package. Un-commenting the following should show you the
problems:

--8<---------------cut here---------------start------------->8---
;; FIXME This is responsible of problems when used with Org-babel
;; (`\LaTeX{}' environments are created around verbatim environments)
;; ;; turn Org blocks into LaTeX environments and HTML divs
;; (try-require 'org-special-blocks)

;; literate programming
(try-require 'org-babel-init)
--8<---------------cut here---------------end--------------->8---

With the above configuration, I could finally export the Org file in LaTeX.
Yeeesss.

I have some comments on the resulting file:

    o   sections (and sub-, and subsub-) should not be starred by default,
        IMHO, and then unnumbered.

    o   indentation of code blocks should be preserved.

    o   name of code blocks should be visible in the produced document
        because, now, I see that the file Enterprise.sql will include the
        sql-init block, but I don't know where it is described (which page,
        and which block).

Except these, the result is already very promising!


>> BTW, what do you think of my proposal to include the Tangle process as a
>> built-in step of the export to LaTeX (and HTML)?
>>
>> That way, we only would export and get both flies caught with one stone.
>
> I like that idea, once this is working as two separate processes it should
> be relatively trivial to optionally hook org-babel-tangle onto the other
> export commands.

Fantastic.

Thanks for all your work.

Best regards,
  Seb

-- 
Sébastien Vauban