Re: [Bug-zile] Incremental search is dog slow in moderately-sized files

[Gary V. Vaughan]

Hi Reuben,

On Jan 21, 2014, at 12:20 PM, Gary V. Vaughan <address@hidden> wrote:
> On Jan 21, 2014, at 12:10 PM, Reuben Thomas <address@hidden> wrote:
> 
>> On 20 January 2014 23:07, Gary V. Vaughan <address@hidden> wrote:
>>> Thanks for taking a look at this.
>>> 
>>> On Jan 21, 2014, at 4:07 AM, Reuben Thomas <address@hidden> wrote:
>>>> 
>>>> I've run out of time for [[debugging the alien.default.memrchr crashes on 
>>>> glib]] for now. However, I have an idea: how about adding asserts to your 
>>>> Lua memrchr to check for out-of-bound accesses? I can't see anything wrong 
>>>> with memrchr itself now, so I suspect bad arguments which merely result in 
>>>> nil results in the Lua version.
>>> 
>>> I've noticed that about half the time, a random one out of the 100+ 
>>> run-lisp-tests.lua checks (`make tests-check-local`) crashes zmacs using my 
>>> Lua memrchr too.  Instrumenting shows that my memrchr returns the location 
>>> of EOL prior to the one returned by find_substr().  Naïvely changing 
>>> memrchr to start the search one byte higher in memory breaks everything 
>>> though, so I need to spend some time understanding the details of 
>>> find_substr, and either fix that and whatever changes ripple out from 
>>> there, or make sure memrchr is interface compatible before swapping it out. 
>>>  At least I have a way of pinning it down now... and then I can make the 
>>> same changes in the alien.default.memrchr wrapper and hope that works too.
>> 
>> When I was translating all this stuff from C to Lua it was almost always an 
>> off-by-one bug in these cases. It's possible I may have left some in the 
>> Zile code myself!
> 
> I spotted many of those fixes in the gitlog.  However, since there is no 
> 0-based indexing anywhere in Lua Zile, I guess these are purely artifacts of 
> the C->Lua transition, right?  And yet there's a ton of +1 and -1 adjustments 
> scattered around that I don't fully understand yet. Or are Zile buffers 
> 0-based, and have to be adjusted to Lua 1-based indices?  If so, then I might 
> try and go for a clean slate by making everything 1-based, except when 
> accepting or displaying buffer indices in the zlisp Defuns...

It took several hours of debugging, but finally I discovered that estr:line 
calls astr:find with out-of-bounds arguments at the end of a loop, expecting to 
get back a `nil`.  Adding guards to find and rfind and returning nil rather 
than letting memchr/memrchr race through random memory outside of the allocated 
buffer has fixed both my occasional crash in find, and the major test breakage 
in memrchr on Travis.  There are probably other instances of that assumption, 
but I guess I'll have a better chance of teasing those out as I spec out the 
apis with Specl in the coming months.

You should be able to copy the memrchr + fallback code into Zee now to get a 
version that works on non-glibc systems :)  I still need to fix it to work with 
CRLF line endings in Zile, but that isn't something Zee supports.

I'd be especially happy if you could confirm `make tests-check-local` mostly 
behaves on Linux with current master.

Cheers,
-- 
Gary V. Vaughan (gary AT gnu DOT org)

signature.asc
Description: Message signed with OpenPGP using GPGMail