Re: [Bug-apl] ISO Component File API preview

[David Lamkins]

Hi Blake,

I feel that we're talking at cross purposes.

What I've been trying to convey (and clearly have failed on every attempt so far) is that an APL component file system must be file-centric. Every APL component file system that I've worked with or have read about has created a file to contain its APL data components.

I know you don't like looking at the ISO document, but please: read Annex A before continuing this conversation. The Annex is concise and informal and describes what a component file system is expected to do.

My use of SQLite is an implementation detail - nothing more - underneath the covers of the component file system implementation. Had Elias never read the code, all you'd know is that my implementation does satisfy the requirements of the design outlined by the Annex of the ISO APL spec.

Please, let's acknowledge that, as a baseline, a component *file* system requires *files*... Look at the component file systems as implemented by STSC, Dyalog, MicroAPL and others: they all store data on files; not database systems.

I feel that strong assertions that this library *must* do things a certain way, when those assertions seem based upon a fundamental misunderstanding of what the library is designed to do, are not constructive. Perhaps I should have been more clear regarding the intent of the library at the start of this conversation; I incorrectly assumed that everyone involved had already familiarized themselves with the reference material and the implementation and was making suggestions about possible futures.

I still believe that it may be useful to allow different database backends as an experimental conforming extension of the component file library. That, however, is not and has never been a design goal. I'm going to insist that we address the correctness and effectiveness of the library relative to the design expressed by the Annex before we proceed further with proposals for extensions.

Thanks for your feedback so far. I do hope you'll make the time to evaluate this library in the context of Annex A of the ISO spec.

On Thu, Jul 10, 2014 at 11:45 AM, Blake McBride <address@hidden> wrote:

Dear David,

I haven't actually had time to look at your work yet.  I am going off of what Elias said, my fair amount of experience with SQL databases, my experience writing the keyed files system, and the comments you have made.  Forgive me if I am wrong, but based on your comments, it seems like you don't have a lot of experience with SQL databases.  

In SQL terminology, there are no 'files'.  A 'database' is analogous to a D: drive in Windows, or a partition, file system, or mount point on Unix.  What you would thing of as a 'file' is called a 'table'.  A database can have any number of tables.  You would never think of putting one table per database.  That would be like creating a new drive for each file.

Each table can have any number of rows (or records).  Each record in a single table will have the same data elements as the other records in that same table - sort of like a spreadsheet.  You can order or select by any row or any combinations of rows.  In practice, however, it is best to have an index setup for the more popular types of queries you will be doing.

SQLite keeps each database in a single Windows or Unix file.  PostgreSQL has a complex internal data representation made up of many Windows/Unix files.  Both of these  facts are totally abstracted away from you, and in fact shouldn't matter to you at all.

So, in implementing a component file system, it makes most sense to equate a single APL component file to a single SQL table.  Each component would be a record in an SQL table.  Ordering and access are handled automatically by the database.  Your implementation should not care much at all whether it is SQLite or PostgreSQL.

I hope this is helpful and not disrespectful.

Sincerely,

Blake

--

"The secret to creativity is knowing how to hide your sources."

   Albert Einstein

http://soundcloud.com/davidlamkins
http://reverbnation.com/lamkins
http://reverbnation.com/lcw
http://lamkins-guitar.com/
http://lamkins.net/
http://successful-lisp.com/