Re: smie-next-sexp vs associative operators

[Stephen Leake]

Stefan Monnier <address@hidden> writes:

>>> That's actually a bug: the THEN should be indented relative to its
>>> corresponding ELSIF (if any) rather than to the top IF.
>> _all_ of the keywords in the if/then/elsif/else/endif, and in every
>> other multi-keyword statement in modula-2 and Ada, work this way; they
>> _all_ jump to the first keyword in the statement. Except for the ones
>> that happen to be associative. You'd have to totally rewrite
>> smie-backward-sexp, to fix this "bug".
>
> SMIE likes to have more structure in its parse tree: instead of an IF
> construct having a single level
>
>     (IF_THEN_ELSIF_THEN_ELSE_END exp1 stmt1 exp2 stmt2 stmt3)
>
> it likes to have something deeper, maybe even as deep as
>
>     (IF_END (_ELSIF/ELSE_ (_THEN_ exp1 stmt1) (_THEN_ exp2 stmt2) (stmt3))
>
> One of the advantages being that if a THEN is missing or there's an
> extra one somewhere, it should still be able to match the END with
> its IF.

Ok, that's clearly a design choice.

It is completely orthogonal to the issue of special treatment for
associative operators; none of these will be associative.

My initial reaction is "if you want to move from one keyword to the
previous, write motion code that does that, don't mess with the
grammar".

Such motion code is possible; another choice for the termination
condition of the loop in smie-next-sexp should do it - stop at the first
keyword with a matching level. That would require a flatter grammar, so
the keywords in one sexp do have matching levels.

>> But all of this just goes to say that the current behavior of
>> smie-backward-sexp for _non_-associative operators is wrong in the first
>> place; that's a _major_ redesign!
>
> I don't see why.  If you make your parse tree deeper, you'll get the
> same "stop at the closest token" behavior.

Ok, that makes sense.

But it means spending time optimizing the grammar so it has these
properties. As I've said, I've got way to much work to do, so keeping it
simple is more important at the moment.

> BTW, this discussion is really useful to make me figure out some of the
> guidelines and design rules that we should document in
> SMIE's documentation.

Well, good. I've been learning more, as well.

>> That is pretty much the crux of the matter; we have different visions
>> about how indentation engines should work, partly influenced by the
>> different languages we are implementing.
>
> I don't think it's a question of language (I've implemented SMIE support
> for Pascal, sh, Modula-2, Prolog, SML/OCaml/Coq, and Octave, which

Are those in emacs trunk? I've only looked in 24.2 so far, and there
only modula and octave use smie. 

All of those are far smaller than Ada, by refined keyword count if no
other measure. Prolog and the SML family are certainly very different
languages. 

> I think covers a fair variety of syntaxes).

Yes.

So I'll fall back on "the default value for smie-skip-associative is
tied to the language implementor". Still seems valid.

> I've just learned to use SMIE to its advantage, whereas you're trying
> to fight it.

That's fair. 

It's also fair to keep things very simple at first, and only optimize
when it becomes necessary, and add features like single-keyword motion
when you have the time.

>>> Can you tell me in terms of C-M-f or C-M-b in which circumstance they
>>> don't behave identically (and show me the corresponding grammar rules
>>> you're using)?
>> Not directly, no, because C-M-f doesn't call (smie-backward-sexp
>> keyword), it calls (smie-backward-sexp 'halfsexp), which gives different
>> behavior.
>
> No, C-M-b gives the same behavior, you just have to start from another
> position (i.e. from right after the keyword rather than from before
> it).

True. 

Given this code (Ada except "end if" replaced by "end fi" to avoid
refining, similarly for "end select"):

   if A then     -- then-1
      B;

   elsif C then  -- elsif-1, then-2
      D;

   elsif         -- elsif-2
     E then      -- then-3
      F;

   else          -- else-1
     G;
   end fi;

   select
       accept A1;
       B1;
    or           -- or-1
       accept A2;
       B2;
    or           -- or-2
       accept A3;

    else         -- else-2
      B4;
    end tceles;

Using grammar-1a, smie-skip-associative nil: 

With point after "fi", C-M-b leaves point before "if"; similarly for
"tceles" and "select".

With point after the "else" labeled else-1, C-M-b leaves point before
"if". For else-2, point is left before "accept A3". For or-2, point is
left before "accept A2".

or-2, else-2 are the only exceptions; after all the other keywords,
C-M-b stops with point before "if" or "select".

Setting smie-skip-associative t removes those exceptions; C-M-b gives
the same behavior after all keywords.

-- 
-- Stephe