Do we really offer the future?

[Urs Liska]

Just one more of the fundamental questions I took home from the 
Musikmesse ...

The question can be asked somewhat less pretentious then in this 
message's subject line, but I think it actually boils down to no less 
than that.

You know that I have again been at the Frankfurt Musikmesse this week, 
and again I had the opportunity to talk with various people from 
publishing houses (names only privately ...), and I was (unpleasantly) 
surprised that I didn't always have fully satisfactory answers ready.

The questions came in various variants of "why should a publishing house 
use LilyPond?" And despite all the reasoning and writing I have produced 
over the last years I didn't always find "the" striking key features 
that were convincing in the concrete situation.

I think  I have always taken a perspective that was focused slightly 
beside the point, namely the perspective of an individual editor or the 
team of editors. This is perfectly transferable to a publishing house 
starting from scratch, but not to a big house with traditions, 
regulations and limitations.

Compared to last year I have the impression that many people have become 
more aware of the basic questions about longevity of binary and textual 
data formats and data processing. The question has become much less "why 
should we consider dropping Finale and Sibelius, it's working, heh?" and 
more "OK, we see that we need an alternative approach, but how do you 
convince me that LilyPond has to be it?"

We always say that text based tools are superior because they are much 
less prone to become unusable. This, and the potentials that come from 
version control, make quite some impression, but again this is only 
relevant to new material and doesn't take into account two important issues:

1)
Publishers receive heterogenuous data material from various sources 
(editors, composers, engravers), and these are mostly done using Finale 
or Sibelius (and in some cases Score). It is completely out of question 
to requre all these people to switch to LilyPond.

2)
One major question publishing houses consider today is how to carry 
their (digitally) existing material to the future. By now they have 
realized that simply using the latest version of the mainstream programs 
is calling for disaster, that's good for us. But again, the question is: 
Can we really offer "the" solution for this?

The immediate idea would be to go some route via MusicXML and offer 
hassle-free workflows to convert existing editions or said received data 
material) to LilyPond. But firstly I don't think we can already 
guarantee this. And secondly, the question is natural to pop up: If one 
already uses MusicXML as the permanent and future-oriented storage 
format, where is then the need to consider LilyPond for this?
I think it is reasonable to expect that there will always be mainstream 
programs that can work with MusicXML files, and their user base will 
probably always be larger than ours.

###
So, now I've somewhat laid out why we are actually facing this question 
about "offering the future".
I can't give convincing answers to that, but of course I have a few 
ideas, and I would be happy about a constructive discussion. This is not 
a pipe dream discussion as I will (have to) pick up the communication 
with the publishers soon. And it will probably make a good impression if 
I can show that we have taken their concerns seriously, especially if I 
can come with some promising suggestions.

The first asset is the fact that plain text tools allow highly 
sophisticated workflows and adaptation of the programs' functionality. 
The biggest impression I could make was probably our "grid" approach (of 
course only backed up by the fact that we have successfully realized it 
in a real project), and - to some extent - the prospect of maintaining 
the whole edition process within one single context (the \annotation 
functionality in the ScholarLY library). But the latter was somewhat 
less striking because it seems most publishing houses don't really care 
anymore about the editorial process, and they have the impression that 
this could actually create more overhead than they have currently.

The second asset I see is that we can (principally, in the real-world it 
isn't completely mature yet) completely separate content from 
representation, which should be stressed very much when it comes to the 
questions of long-term data storage and of repurposing content.

In theory, LilyPond (as LaTeX) is better suited to process textual data 
than binary tools, simply because it's their natural appraoch. But I 
don't know what I would answer to the question "well, yes, I see, but 
what *impact* does this really have?"

There is one road that I could see as the "golden bridge".
I think MusicXML isn't the best solution for long-term management of 
editions. It's just too much focused on data exchanged between music 
software - and mostly pop music oriented tools. Looking for a 
fundamental solution to migrate the whole data base of a publishing 
house I would think that MEI is a much better solution because it's 
inherently more comprehensive, and because it has been originally 
conceived from an editor's perspective rather than music production tools.
Currently there doesn't exist *any* straightforward way to render MEI 
encoded data with a professional engraving program, so this could be the 
key feature for avoiding the question "so why should we prefer LilyPond 
over Sibelius or the new Steinberg app?".
IF we could come up with a promising path to let LilyPond work with MEI 
data (that is firstly: use MEI as input to LilyPond and/or convert MEI 
data to LilyPond files, and secondly: Be able to convert to both 
directions so one can also edit scores as LilyPond and convert them back 
to MEI for storage) that _could_ be the satisfactory answer I claimed as 
missing above. Publishers have more than once thought about a concerted 
effort with regard to data management. Of course that would also imply a 
platform for _distributing_ music, so the option/request to provide 
*digital* scores is also ubiquituous. Actually this was again suggested 
this week in one of my meetings, and I would love to pick that up with a 
more or less concrete but at least convincing outline.
At the same time the prospect of publishers (or "the" publishers in a 
common effort) would consider MEI this would motivate the MEI community, 
and if it would be somehow connected with LilyPond it would raise the 
chance that some of them would actually step out and start something 
worthwile (I know there is a latent interst in LilyPond within the MEI 
community, but there's no sufficient "market" for it so nobody actually 
came up with a solution so far.


####
####
To conclude:

- most people in the business have moved away from taken the status quo
  with Finale and Sibelius for granted.
- they know that they *have* to find new answers.
- many (except a few die-hard reactionists) see that LilyPond and 
friends *can* offer answers to their questions
- but they also see that these are maybe not the only possible answers and
- that we (currently) can't guarantee straightforward migration paths.

Market is hard, and everything is moving quite slowly, of course.
But IF we should be able to come up with convincing solutions or at 
least roadmaps I see that we now have better chances than ever to get 
LilyPond a foot in the door with the publishing business in general.

Sorry for that elaborate text, but I think it is important and hopefully 
fruitful.

Best
Urs