bug#11974: 24.1.50; python-nav-forward-sexp-function doesn't skip over strings

[Ivan Andrus]

Oops, forgot to cc the buglist.

On Jul 19, 2012, at 9:52 AM, Stefan Monnier wrote:

>> (let* ((bos (python-info-ppss-context 'string)))
>> (goto-char bos)
>> (forward-sexp 1)
>> (narrow-to-region bos (point)))
> 
>> This is broken now since forward-sexp goes to the end of the block, far
>> past the end of the string (or not moving at all).  If you have any
>> undindented code at the end of your buffer, then forward-sexp (and up-list)
>> will refuse to move past it e.g.
> 
> There's a similar problem in SMIE's implementation of forward-sexp.
> Basically, the issue is that there are several "sexp"s that start at
> point and forward-sexp has to choose which one to skip over.
> Every choice is valid, but since there's no way for the user to say
> which choice she wants, any choice is bound to disappoint and/or
> surprise the user sometimes.

I didn't realize SMIE implemented forward-sexp.  Yet another reason for me to 
use it.

> That doesn't mean that it's a hopeless problem and that we shouldn't try
> to improve the behavior, tho: in practice (maybe just for historical
> reasons), I find that I generally prefer "the smallest sexp", especially
> since I can repeat forward-sexp if it didn't skip enough.

I agree.  The smallest is usually best.

> I.e. Python's forward-sexp should only try to skip over
> indentation-delimited blocks when it has a clear indication that only
> skipping the current line/statement would be wrong, or that there's an
> easy/intuitive way for the user to get the other behavior (e.g. by
> starting the forward-sexp from elsewhere).
> 
> For example, with SMIE if I have
>  a + b * c
> and I'm before "b", forward-sexp will only skip over "b", but if I move
> to just before the "+", then forward-sexp will skip to after "c".
> I'm not sure if Python's indentation-delimited blocks can find a similar
> "convention" for the user to express her intention.
> 
>> What I would expect form a "perfect" python forward-sexp is to skip over
>> a block if and only if you are at the very beginning of the block.
> 
> You mean that forward-sexp would generally skip over just a statement,
> except if it's the first statement of a block in which case it skips the
> whole block?  That sounds a bit arbitrary, making it difficult for the
> user to skip over just the first statement.

No, it would only skip over a block if it's at the block opening.  In some 
sense the entire block is one statement hence my thinking.  See below.

> [ Bear in mind, I never write Python.  ]
> How 'bout
> 
> def foo():
>     toto
>     bar
> 
> if I'm before `toto' on the same line, forward-sexp should skip over
> "toto", but if I'm right after the ":" (and before the \n) then
> forward-sexp should skip to after "bar".

In my mind you should skip to after bar only if the cursor is before the def 
(or possibly anywhere in the def keyword).  If I'm anywhere after that I 
consider myself to be "inside" the block.  But that's only the way I think and 
may not be representative of other (more serious) python programmers etc. etc.

I guess I would treat both if and else as separate "block openers" rather than 
if opening a single block.  Maybe an easy rule would be if you are at the 
beginning of the statement (well between the end of the last and the beginning 
of the current) go to the next statement that has the same indentation?  
Anywhere else do as a "normal" forward-sexp.

-Ivan