Re: [Bug-apl] Elias' SQL code

[David B. Lamkins]

I wasn't previously aware that an appendix of the ISO standard called
for a component file system. Thanks for calling that to my attention.

I based my API on a vague recollection of component file systems of days
past. (In particular, STSC's component files as implemented in Richard
Smith's short-lived APL*PLUS/Macintosh...) I haven't been able to locate
much on the web regarding that particular API, although Micro-APL has an
API that looks very similar to what I remember as the STSC API.

It turns out that, as currently implemented, the API of my component
file system doesn't match the API described in the ISO appendix. There
are some coincidental similarities, but the details differ. 

I'd certainly be willing to make my API conform to that of the ISO
standard.

My primary concern with the sqlite-based component files is that
performance is, by default, not great. SQLite imposes a transaction by
default; the default transaction creates a performance bottleneck on the
order of 10 operations per second. On the one hand, the transaction
guarantees that any updates are either completed or abandoned; the
component file is never left in an inconsistent state. On the other
hand, 10 operations per second is nowhere near being impressive.

The flip side of this is that one can specify explicit transactions to
batch a number of updates. If you take a look at the performance test in
my component files package, you'll see that this scales up very quickly.
It seems that batching operations within a transaction can easily get a
couple orders of magnitude improvement in performance per operation; the
limit seems to be around three orders of magnitude improvement over the
one-operation-per-transaction case.

I'm not really pleased with my implementation of transactions, which
passes the operation to a APL operator. You have to do some
unnecessarily clever construction of the operation function. I think I
might restructure this portion of the API as a transaction begin/end
pair. Stay tuned for future developments.

Regarding the related discussion of Blake's keyed files, I'd prefer that
these retain the current KF prefix. Component files historically have a
very narrow definition of using only integer keys. Keyed files should be
treated as distinct from component files.

On Sun, 2014-07-06 at 08:47 -0500, Blake McBride wrote:
> Dear Juergen,
> 
> 
> While the keyed file system is often more desired than a component
> file system, the semantics of the two are different enough that one
> cannot easily substitute for the other.  Having both options would be
> a good thing IMO.
> 
> 
> David wasn't part of our prior conversation on having the component
> file system use the same function names and semantics as that
> described in Annex A of the APL standard.  Perhaps he can do that.
>  David would have to speak to his component file system.
> 
> 
> Do you have interest in including the keyed files system, or shall we
> wait for now?
> 
> 
> Thanks.
> 
> 
> Blake
> 
> 
> 
> 
> On Sun, Jul 6, 2014 at 8:40 AM, Juergen Sauermann
> <address@hidden> wrote:
>         Hi Blake,
>         
>         yes, sorry. The bug-apl mailing list just passed the 2000'th
>         email and
>         I did not yet have the time to fully understand the different
>         contributions
>         related to file systems.
>         
>         When I briefly read the ISO standard I thought an SQL database
>         with integers as keys and 10 ⎕CR of
>         a value would do what they want. I may be wrong, though.
>         
>         Anyhow, what I am after is to have an ISO compliant component
>         file system that can be
>         shipped with GNU APL(maybe David's is compliant already?)
>         
>         /// Jürgen
>         
>         
>         
>         
>         
>         
>         
>         On 07/06/2014 03:12 PM, Blake McBride wrote:
>         
>                 Dear Juergen,
>                 
>                 Looking at Annex A, I am reminded that that has to do
>                 with component files not keyed files.  I think David
>                 did a component file system (I think we should add
>                 too).
>                 
>                 All of the component file system functions begin with
>                 CF_.  Perhaps I should change my prefix from KF∆.
>                  Arguably, KF∆ is a better prefix than KF_ because the
>                 underscore is used to separate words.  The delta is
>                 used to separate packages.  All this by some sort of
>                 pseudo standard (perhaps only in my mind).
>                 
>                 What would you prefer?
>                 
>                 On another note, I am familiar with TeX and Texinfo (&
>                 troff).  If you think it helpful, I can convert and
>                 enhance the docs to texinfo format.
>                 
>                 Thanks.
>                 
>                 Blake
>                 
>                 
>                 
>                 
>         
> 
>