bug#10072: 23.3; invisible text

[Stefan Monnier]

> Sorry, but I didn't receive this latest version.  Could you re-send it?

Sorry, it's appended this time.

> Right this moment, I don't have access to the latest version
> of Emacs, so I am not sure of the relevance of this remark:
> However, I was experimenting with invisibility in overlays
> and I found it was possible to insert invisible text.

> I suspect that the philosophy of the command-loop-move-point
> thingy is to move point so that invisible text will not be
> inserted in /any/ case.

No: doing it reliably is fiendishly difficult and in general cannot be
done without breaking some code somewhere.  So instead the code does
a best effort which covers 99% of the cases.  It's fundamentally
very DWIMish.

> So there's another possible bug.

Probably, but maybe it's one of those cases where we can't win.

>> Fundamentally it's a bug, and I generally don't like to document bugs.
> On this point I disagree with you very strongly.  It's much /more/
> important to document bugs than any other aspect of the code.  I have
> told several students this from time to time, and, like you, they
> tend to resist the idea.

I tend to consider Emacs as a great big pile of bugs.  It very much
follows a pragmatic "don't worry too much about corner cases" (some
people might call it "worse is better"), so documenting all bugs would
be a daunting task.  You can see the bug-tracker as a way to document
the known bugs.

> I gather that there is no great push on to remedy this behavior,
> so this bug may hang around for some time . . .

Yes, I don't know of any effort to solve it, because the inconvenients
tend to be rather minor compared to the effort it would take to make it
work more reliably (some of those efforts would be spent figuring out
what the hell Emacs should do in various corner cases).

I'd like to see a pre-redisplay-functions hook added to Emacs for
various reasons (e.g., for reveal-mode, as well as to move the
region-highlighting code to Elisp), and such a hook might possibly allow
a more reliable implementation of this feature, but don't hold
your breath.

> Anyhow, I'm pretty happy with my improved understanding, so I'm
> grateful for your help.  Are you a volunteer?

Yes, like all the other Emacs developers.

> I guess that FSF doesn't have enough money to employ a staff of
> programmers to deal with this sort of correspondence.

Indeed.


        Stefan


=== modified file 'doc/lispref/display.texi'
--- a/doc/lispref/display.texi  2011-11-20 02:29:42 +0000
+++ b/doc/lispref/display.texi  2011-11-20 20:21:22 +0000
@@ -870,15 +870,21 @@
 non-@code{nil} (the default), but only because they are explicitly
 programmed to do so.
 
-  However, if a command ends with point inside or immediately before
-invisible text, the main editing loop moves point further forward or
-further backward (in the same direction that the command already moved
-it) until that condition is no longer true.  Thus, if the command
-moved point back into an invisible range, Emacs moves point back to
-the beginning of that range, and then back one more character.  If the
-command moved point forward into an invisible range, Emacs moves point
-forward up to the first visible character that follows the invisible
-text.
+  However, if a command ends with point inside or at the boundary of invisible
+text, the main editing loop moves point to one of the two ends of the invisible
+text.  Which end to move to is chosen based on the following factors: make sure
+that the overall movement of the command is still in the same direction, and
+prefer a position where an inserted char would not inherit the @code{invisible}
+property.  Additionally, if the text is not replaced by an ellipsis and the
+command only moved within the invisible text, then point is moved one extra
+character so as to try and reflect the command's movement by a visible movement
+of the cursor.
+
+  Thus, if the command moved point back to an invisible range (with the usual
+stickiness), Emacs moves point back to the beginning of that range.  If the
+command moved point forward into an invisible range, Emacs moves point forward
+to the first visible character that follows the invisible text and then forward
+one more character.
 
   Incremental search can make invisible overlays visible temporarily
 and/or permanently when a match includes invisible text.  To enable