Re: keep information of texinfo source

[Patrice Dumas]

On Sat, Dec 10, 2011 at 01:35:42PM +0100, Patrice Dumas wrote:
> Hello,
> 
> But there is also some information lost, such as protection of end of lines
> in @def* commands with @, or @include file beginning or end, user defined
> macro and value expansion start and end, cpp synchronisation lines, \x{7F}
> comments, @if* commands both for kept and removed blocks.

Thinking a bit more about that, I have come to the conclusion that this
is not that easy.  There are 2 issues, one is the possibility that macro
do not fit at all in the tree (even not balanced).  This shouldn't be that
frequent but could happen.  For example

@macro test
A @commmand
@end macro

@text{}{a command}

The end of the test macro happens between the command name and the argument.
It does not fit in the tree.

There are other cases, like text generated in special commands that are
not interpreted as texinfo code but as raw text.  For example, @set:

@macro testset
@set ggg
@end macro

@testset blah @code{bloh}

@value{ggg}

Problem here is that 'blah @code{bloh}', the @set value is interpreted
as text, and therefore it is as is in the tree:

      'cmdname' => 'set',
      'args' => [
        {
          'text' => 'ggg',
          'type' => 'misc_arg'
        },
        {
          'text' => 'blah @code{bloh}',
          'type' => 'misc_arg'
        }

There is a less problematic case, with the end of a user-defined macro,
end of include file or @end if* in a format where Texinfo code is ignored.
For example

@macro mymacro
@verbatim
@@@@@@@
@end macro

@mymacro{}
@@@@@@@
@end verbatim

The end of macro expansion marker happens in a @verbatim.  This is not as
bad as above, as lines of @verbatim ar text elements in the tree, so it is
possible to split text elements and insert the marksource between them.
But the simpler implemetation in which a @marksource is put at the end of the
macro expansion will fail because it will be in a @verbatim block where 
texinfo code is not expanded as such.  It could be possible to use another
marker, though, like \x{7F}, but I am not sure we want to go there.

It is also not easy to detect that we are in such cases.

At that point, I think that I will proceed with the plan, as those cases
should be rare anyway, but we should document the limitation of the tools
that rewrite Texinfo code.

Agreed? (of course all that is for after the release).

-- 
Pat