Re: Needed: per-port reader options

[Ludovic Courtès]

Hi Mark,

Mark H Weaver <address@hidden> skribis:

> address@hidden (Ludovic Courtès) writes:
>> Mark H Weaver <address@hidden> skribis:
>>
>>> I recently tried to implement reader directives, e.g.:
>>>
>>>   #!fold-case     (from R7RS)
>>>   #!no-fold-case  (from R7RS)
>>>   #!curly-infix   (from SRFI-105)
>>>
>>> However, I ran into a rather serious problem.  Guile's reader options
>>> are global to the entire process, but clearly these reader directives
>>> should affect only the port they are read from.  So we need to support
>>> per-port reader options.
>>
>> I think we should instead be able to instantiate new readers–i.e., have
>> a higher-order ‘make-reader’ procedure (that’s how Guile-Reader started,
>> actually.)
>>
>> That, combined with the ‘current-reader’ fluid, should provide good
>> flexibility.
>
> Being able to easily create new readers sounds great to me, but that's
> not sufficient to implement the reader directives above,

Of course.  I just meant that, if you can call ‘make-reader’ with
whatever options you’re interested in, instead of globally changing the
reader’s option as is currently the case, then you’re halfway through.

And in that case, the reader doesn’t need to be associated with the
port.  Instead, since ‘primitive-load’ honors ‘current-reader’,  it just
needs to be set when loading a file.  This way, any changes to the
reader’s options will be local to that file.

Yet, this avoids the need to explicitly have a port-to-reader mapping.

(That’s how I’ve used Guile-Reader in Skribilo, FWIW.)

Concretely, this would mean changing read.c such that each token reader
takes the reader options as an additional first parameter.  Instead of
looking up the global ‘scm_read_opts’, they would look at this explicit
parameter.

How does that sound?

> the first two of which are mandated by both R6RS and R7RS (draft).
>
> Those standards mandate that we be able to change the case-sensitivity
> flag on a per-port basis, while in the *middle* of reading a datum,
> anywhere that comments are permitted.
>
> Suppose we ignore those standards requirements.  If someone wants to
> distribute a portable module with case-insensitive symbols or
> curly-infix syntax, how would you recommend that they portably arrange
> for their file to be read by a special reader?
>
> Even if you ignore these problems (which are bad enough), there's also
> the issue that one must reimplement the entire reader in order to add
> one new feature.  This means reimplementing things like Guile's array
> syntax (which is quite hairy), somehow hooking into 'read-hash-extend',
> and keeping all of these reader implementations in sync with each other.
>
> SRFI-105 is actually quite simple to implement, and in my latest patch
> it required modification in only a few places.

I agree we need these reader extensions, of course.  I’m just wondering
about how to do it best.

>> Also, I think ports should remain reader-oblivious.  ‘read’ is
>> fundamentally at a higher level of abstraction, so I’d personally prefer
>> not to have ports fiddle with it.
>
> I agree.  In my patch, ports don't know anything about read.  I merely
> made the values of 'scm_i_port_weak_hash' into alists.  Previously their
> values were unused, and always #f.  In Guile 2.2, I'd like to move this
> alist into the port structure itself for better efficiency.  The
> built-in reader just adds an item to this alist if per-port read options
> are set.
>
> What do you think?

I think ‘current-reader’ should remove the need to have a port-to-reader
mapping, no?

Thanks for looking into this!

Ludo’.