Extending LilyPond with packages

[Urs Liska]

I'm starting a new thread because the original thread has diversified too much already.

Am 05.12.2013 18:48, schrieb David Kastrup:

For starters, we could take
>https://github.com/openlilylib/snippets/tree/master/templates/predefined-instruments
> and expand it, and add such predefined "instruments" to official
> LilyPond.  I think it would make "structural" work much easier (esp.
> for beginners).

We need to figure out how we can provide "style sheets", similar to how
LaTeX makes it possible to define "document classes" (layout definitions
and tools) and "packages" (raw functionality packaged into coherent
interfaces).

Moving in the direction where this is possible also takes some pressure
of stable/unstable development and features/fixes: something which comes
in its own, optionally used file is not disruptive to the core
stability.

OK, I'll give it a first try.
Please be aware that this isn't a specification put up for review but a first shot to nourish a discussion.
As a working hypothesis I'll try to take LaTeX and TeXLive as a model.

Adding the concept of "document classes" and "packages/stylesheets" to LilyPond seems a very good idea to me. This would allow a consistent way to add functionality to LilyPond without a) bloating or b) compromising it.
Responsibility for stability and usefulness would be deferred to the package author and users, code is only loaded into LilyPond when the user decides to need it in a concrete score. I don't think it's a problem that users may run into "command not found" problems. This equally present in LaTeX, and it isn't really a problem.
Consequently this would allow us to be much more generous when it comes to accept contributions. CTAN respectively TeXLive more or less accept any contribution if it
- is suitably licensed
- has documentation (not only in binary form)
- doesn't create naming conflicts.
There is practically no selection based on quality of code, concept or being in line with the preferences of the maintainers.
I think this could be copied.

For now I'm concentrating on packages because document classes seem more complex, and one aspect should be enough for a first post.

----

Provide a directory structure in the LilyPond directory. This is automatically included in the search path and will be searched recursively, so I can simply use "myNewPackage" although it's actually in the "fancy/engravers" subdirectory.

Maybe it would be good to consequently accept a parallel "private" directory structure (like ~/texmf) where the user can manage his own packages or 'install' packages received from others or downloaded somewhere.
This isn't necessary because it could easily be dealt with through traditional search paths, but I'd say it's a nice way to get things consistent.
If a package isn't found in LilyPond itself search
- a default location (e.g. a directory in the homedir)
- look for an environment variable
- an include path (or all) specified on the command line

---

One question is whether the library should be included in the default installation/download or if it should be made optional.
Consequently one would have to discuss if one needs a package manager like tlmgr.

---

Basically a "package" can provide arbitrary code to be included.
It can for example provide
-  overrides to modify the output appearance (classic "stylesheet")
- new commands / functionality
- alternative header definitions with arbitrary header fields
- engravers or anything one would otherwise include too.

Provide commands like \usepackage and \RequirePackage (naming could be discussed) that include the package file.
Differences to \include
- search path management (recursive search)
- allow options:
  e.g. (in LaTeX syntax)
  - \usepackage[font="Libre Baskerville"]{mySongbookHeader}
  - \usepackage[staffsize=17]{beamerLayout}
  - \usepackage[keep-only-original]{originalBreaks} % ;-[]
  - \usepackage[console,color,html]{annotate} % (print messages to console, color grobs, export html list)

Optional arguments could be passed as a Scheme alist.

Maybe it would be good to provide commands to consistently handle such options inside the package so one doesn't have to reinvent the wheel in each new package.

---

A package should have its own manual like in LaTeX.
Integration a potentially high number of manuals in LilyPond's documentations seems not maintainable.

---

I think one wouldn't need much more as an interface for packages.
Some of this is also applicable for document classes. But I won't think about this right now although I can imagine them to be extremely useful.

Best
Urs