Improving the API for read options

[Mark H Weaver]

Hi Ludovic,

address@hidden (Ludovic Courtès) writes:
>> I've been avoiding adding a public API for this, because I feel that the
>> current 'read-options' API is poorly-designed and I'd rather take the
>> opportunity to come up with a clean design.
>
> What about just mapping the existing ‘read-options’ to something like:
>
>   (set-port-read-options! PORT OPTIONS)
>
> where OPTIONS would be a list of symbols, as for ‘read-options’?  This
> seems to me like the obvious API extension, but maybe I’m overlooking
> something.

Yes, that would be the obvious minimal change to the existing API, but
there are some problems with the existing API that I'd prefer not to
propagate the existing API as-is.

One problem I have with the existing API has to do with its treatment of
square brackets.  There are multiple things that one might want to do
with square brackets, but since 'square-brackets' is a boolean option,
there is no nice way to add additional meanings of square brackets.  We
could add more boolean options, but that raises the question of what to
do when multiple booleans pertaining to square brackets are enabled.
Which one takes precedence?  And there will forever be this wart on the
API that one particular use has the privileged name 'square-brackets'.

One might make a similar complaint about the 'curly-infix' option, but
in that case it's not so simple, because curly-infix affects more than
just curly braces.  Although it allows us to do (almost) whatever we
want to square brackets, it requires that square brackets be delimiters,
and furthermore assigns a meaning to square brackets in one particular
context: within curly braces, when the open square bracket immediately
follows another expression without intervening whitespace.

This makes me wonder if it might be worthwhile to introduce an
additional layer of abstraction between the read options that the user
sees and manipulates, and the low-level read options that the reader
consults.  It might make sense to make options like 'curly-infix' or
'kawa-style-square-brackets' into higher-level views on the low-level
options.

This in turn suggests that the existing API that allows the user to
query the complete list of read options (which correspond precisely to
the options that they directly set) is a poor model.

We might take a lesson from Scheme's treatment of types.  In particular,
Scheme does not provide a way to query the type of an object.  Although
the Scheme standards guarantee that a certain set of types are disjoint,
in general the philosophy is that an object might be of more than one
type.  Therefore, only type predicates are provided, with the
understanding that more than one type predicate might return true for a
given object.

I haven't yet had time to think this through, but my gut instinct is
that I would prefer an API closer to this:

* We provide an opaque type 'read-options' which we reserve the right to
  change at any time, and a set of predefined read-options objects such
  as 'inherit-all-read-options', 'guile-default-read-options',
  'r6rs-read-options', etc.

* We provide procedures to set! and retrieve the read-options object
  to/from a port, and perhaps to/from a global setting.  We might also
  provide a parameter.

* For each user-visible (high-level) read option, we provide a set of
  predicates and mutators for 'read-options' objects.

* We provide a way to explicitly pass a 'read-options' object to the
  reader.

* We define the order in which the 'read-options' objects are checked,
  e.g. first from the explicit parameter (if any), then the per-port
  options, then the fluid (if we add one), then the globals.

* We provide a way for user-defined token readers to accept the
  'read-options' object from the reader, so that it can be propagated
  properly to subordinate readers.

* We need to think about whether (and how) to expose inheritance to the
  user.  We might want to provide ways to reset an option to "inherit"
  mode and to test whether an option is in "inherit" mode.  However, in
  other contexts the user may just want to know about the applicable
  option.

This kind of API would give us more freedom to enhance and generalize
the reader in the future, while providing an easy-to-use and stable API
that users can rely upon.

Obviously, churning through all these issues is a big job, and I don't
think it's realistic to do this job properly in time for 2.0.7.
Therefore, I'd prefer to avoid exposing this API for now.

What do you think?

    Mark