Re: GDL2/EOF scaling (was: Re: GNUstep roadmap)

[Patrick Coskren]

On Tuesday, October 28, 2003, at 03:11 PM, Helge Hess wrote:

> On 28.10.2003, at 20:52, Patrick Coskren wrote:
> ...
> > > If you have 50.000 person entries with 150.000 address entries you 
> > > just cannot afford to load all that as objects into memory just 
> > > because you want to display "name", "city" and "street". Objects 
> > > have a deep and inherent mismatch with the set approach taken by 
> > > RDBMS.
> > If you've got an app that's loading all those entries at once, then 
> > it's a batch app.
>
> It is an absolutely usual situation that you display some few 
> attributes of a large table and has nothing to do with batch 
> processing (which is working on the whole set of the data).

I was responding to the "50,000 person and 150,000 address" bit.  
That's a whole set of data, and I stand by my statement.  If you're 
just talking about having a table with 50 columns where you only want 
to display name, city, and street, then yes, that's a completely normal 
situation, which EOF handles with aplomb: just create another entity.
If you want to call that a hack, fine: I call it a technique.  Just 
create another entity from the handy menu, point it at the table, 
select "name, city, street", boom.  Done.  Scales great.

> The mail I write will end up in an IMAP4 folder which has at least 
> 10000 entries and the UI can deal with such amounts of data just fine 
> (since it isn't loading the whole mail but rather queries the subject, 
> the sender and the send date from my IMAP server).

EOF will do that just fine: you just have more than one entity mapped 
to your table, one for each context you use it in (listing mail vs. 
reading mail).

> > And EOF isn't designed for batch processing.  You're right as far as 
> > that goes.
>
> I never claimed that. I'm talking about regular database applications 
> and database applications imply that you are dealing with large tables 
> (and in the usual case not display just one or two of them at the same 
> time ...).

I think there's a terminology issue here.  I thought that by "large 
tables" you mean tables with lots of data in them.  I'm beginning to 
think that's not the case and you mean tables with lots of columns in 
them.  Is that right?

> > But in any end-user app, you're only loading a subset of the rows 
> > (and probably a small subset at that) into memory.  So the object 
> > paradigm of EOF works very well.
>
> No it doesn't. While loading 10000+ rows with 3 or 4 attributes of a 
> joined person and address table is a non issue, loading 10000 *full* 
> objects is an unacceptable overhead.
> So you either go down to adaptor level or hack-around the issue, eg by 
> introducing artificial entities which break the whole object 
> encapsulation and references and is awkward for other reasons as well.

No, it isn't.  It almost sounds like you have the idea that Entity = 
Table.  That's just not true, and that's by design: there's nothing 
hackish about it.  You can have more than one entity per table, or more 
than one table per entity.  EOF handles the mapping under the hood.

Sure, there are times when, for performance reasons, you want to load 
raw rows.  In that case, you're right that EOF isn't really doing 
anything you couldn't do with JDBC.  But my point is that such cases 
are much less common than you're making them out to be.

> And this is just because you want to do a "select a,b,c from d,e where 
> d.pkey=e.foreignkey".

Which is what EOF is doing for you under the hood.  It's always a good 
idea to run your app once with SQL logging turned on to look for easy 
points of optimization.

> > Personally, I found EOF no harder to learn than JDBC, and I was a 
> > hell of a lot more productive with it.  YMMV, I suppose.
>
> I didn't deny that EOF lower levels are less productive than JDBC.

Hmmm, I think you misunderstand me.  EOF lower levels are roughly as 
productive as JDBC, IMHO.  But the higher levels rock, and everything 
you've mentioned so far can be easily done at the higher levels.

> But the reverse isn't really true either (well, JDBC is hit by the 
> fact that you need to use it in conjunction with Java ;-)

Well, leaving aside the language issue...  :-)

> But to repeat myself: if you are fine with EOF, excellent :-) I have 
> no mission to make EOF look bad. It isn't, its very cool technology 
> and I love working on that. Yet I personally do not believe that it 
> gives anything for the majority of the people - its certainly not 
> killer.

And this is the core of our disagreement: I think EOF is fantastic, 
downright killer, for the majority of applications.  But it seems our 
differing opinions stem from an architecture that I think is elegant 
and you think is hackish, and that's probably at root an aesthetic 
thing.

-Patrick