Re: GNU Zile ideas for GSoC projects

[Gary V. Vaughan]

On Feb 20, 2014, at 8:21 AM, Reuben Thomas <address@hidden> wrote:

> Could you add more stuff [[...]] and send this to address@hidden ?

Here you go.

<h2><a href="http://www.gnu.org/software/zile/>Zile</h2>

<p>GNU Zile (short for "Zile Implements Lua Editors") is a toolkit for building 
editors. Zile has all of Emacs's basic editing features: it is 8-bit clean 
(though it currently lacks Unicode support), and the number of editing buffers 
and windows is only limited by available memory and screen space respectively. 
Syntax coloring, registers, minibuffer completion and auto-fill are available. 
Function and variable names are identical with Emacs's.</p>

<p>Zile currently currently comes with two editors: Zmacs, which as the name 
suggests emulates Emacs, and Zz, which does not support any form of Lisp, and 
is Emacs-inspired rather than a strict clone.</p>

<p>In approximate order of increasing difficulty, here are some projects that 
would help improve Zile:</p>

<h3>Add Unicode support</h3>

<p>For Zile to be useful for building serious editors, it needs Unicode 
support. Emacs's interface to encoding systems could be used as a model.  There 
are already a few unicode helper libraries for Lua, which should serve as the 
basis for a unicode support module for Zile.</p>

<h3>BDD/TDD</h3>

<p>Zile has a legacy test suite for Emacs lisp compatibility, and a new 
infrastructure for a more comprehensive Behaviour Driven Development suite.  
The legacy tests need to be moved to the new infrastructure, and then the 
existing modules should be formally specified with new BDD examples.  This is a 
great opportunity to get some practice working with TDD tools in a real 
project.</p>

<h3>Idiomatic Lua</h3>

<p>Lua Zile is still mostly a line-by-line translation of the earlier C Zile 
implementation, and as such fails to make the most of what the Lua language 
offers to simplify and shorten high level code, while being too liberal with 
the global namespace and not keeping proper cohesion and coupling between the 
potentially self-contained parts of the code.  Refactoring is already well 
underway, but there's a lot of scope for re-engineering and refactoring the 
rest of the code base to support easier reuse</p>

<h3>Syntax Highlighting</h3>

<p>An earlier fork of Lua Zile provides the proof of concept for a 
barely-fast-enough restartable syntax parser and highlighting engine based on 
regexps and a state-machine.  The new combined multi-editor tree needs to be 
refactored to allow merging the syntax highlighting module.  If time allows, 
the restartable regexp parser would benefit hugely by being rewritten as using 
LPEG (Parsed Expression Grammars), and parsers for languages other than Lua and 
C added.</p>

<h3>Nested Buffers</h3>

<p>The syntax highlighting proof of concept is engineered to allow different 
parts of a buffer to be highlighted separately, for instance an HTML file with 
embedded PHP, CSS and javascript.  This concept should be expanded to allow 
different keycaps to be active for certain parts of a buffer, for instance a 
shell interaction buffer that recognizes compiler error messages and in that 
part of the buffer treats an ENTER keypress as jumping to the file and line of 
the error message.  If time allows, this feature could be improved to display 
of parts of other buffers inside the current buffer, highlighted appropriately, 
in this case injecting the text of the just the function surrounding the error 
message directly into the current buffer, highlighting it correctly according 
to the language it is written in, and then saving it back correctly to the 
appropriate file and removing the injected text from the current buffer.</p> 

<h3>Splay Ropes Buffers</h3>

<p>Right now, Zile provides only a single buffer-gap based buffer 
implementation - as used by GNU Emacs.  A more comprehensive Editor Building 
Toolkit really needs some alternative data-structures for editor builders to 
select, such as a Splay Ropes data structure.  Other sensible data-structures 
as alternatives or additions to buffer-gap would also be welcome.</p>

<h3>Zile Lisp</h3>

<p>The Lisp engine used to simulate a tiny subset of zmacs' elisp emulation has 
seen dramatic improvements in recent months, and already supports many features 
not available in the earlier C Zile project, such as macros, eval/apply, a 
REPL; as well as better elisp compatibility to provide minimal features for the 
BDD specs.  There is a huge amount of scope for improving the Lisp engine 
itself, adding a faster Cons/List implementation and the like, and also for 
adding better elisp compatibility to facilitate porting more of the zmacs 
implementation itself to lisp.</p>

<h3>Zz</h3>

<p>Zz, the other editor currently shipped with Zile, aims to leverage the 
already existing and well proven UNIX tools, as compared to Zmacs which wants 
to be a Lisp shell and IDE in the vein of Emacs.  The current implementation 
has only gone so far in that direction as to replace Lisp with Lua.  Much more 
work is required to replace remaining Emacs-like features with implementations 
that leverage the system tools.  Much inspiration can be gained from Sam and 
ACME on Plan 9; Wily on Linux; and TextMate on Mac OS X... all of which 
integrate with the operating system userspace rather than reimplement 
equivalent features in the editor itself.</p>

<h3>Zi</h3>

<p>Zile currently provides a Lua implementation of a micro-emacs, called Zmacs. 
 To improve the flexibility of the new Lua Zile frameworks, the components 
should be refactored and assembled into an alternative modal vi like editor -- 
which we'll call Zi.</p>

<h3>Graphical Toolkit</h3>

<p>Some care was taken to maintain separation of concerns between the user 
interface (keyboard input and buffer redisplay) and the internals of Zile based 
editors, but with only a single curses user interface it's hard to be sure that 
the abstractions are sufficiently clean.  Adding support for an alternative GUI 
using GTK+, Qt or even the Mac or Windows native toolkits, along with whatever 
refactorings are required to support that would make using Zile to implement 
future editors with different io requirements much easier.</p>

<h3>More Editors</h3>

<p>In addition to Zmacs and Zz in the current Zile release, plus Zi on the 
horizon, there is also a fork of the old single editor tree called Zee, a much 
more minimal editor, which should be merged back in as an alternative editor 
example in the new multi-editor tree.  Further, implementations of Nano, or 
WordStar, or some ancient DOS editor could be added.  Much of the work in 
adding an editor will be to refactor the code on the boundaries between the 
Zile objects and files and the existing editors to facilitate easily writing 
new editors, while keeping the existing ones working correctly.</p>

<p>If you're interested in portable software engineering and re-engineering, 
Emacs or other text editors, or traditional UNIX tools and their future 
incarnations, you should find a project to interest you.</p>

<p><strong>Contact:</strong> <a 
href="mailto:address@hidden";>address@hidden</a></p>

signature.asc
Description: Message signed with OpenPGP using GPGMail