[Top][All Lists]

[Date Prev][Date Next][Thread Prev][Thread Next][Date Index][Thread Index]

Re: [Demexp-dev] How to get through the network an unknown amount of dat

From: David MENTRE
Subject: Re: [Demexp-dev] How to get through the network an unknown amount of data?
Date: Thu, 09 Sep 2004 19:44:46 +0200
User-agent: Gnus/5.1006 (Gnus v5.10.6) Emacs/21.3 (gnu/linux)

Hello John,

skaller <address@hidden> writes:

>> One obvious answer would be to use an API similar to Unix API to list
>> files in a directory:
>>  - a function start_listing() to start the listing;
>>  - a function continue_listing() to have the let's say next 10
>>    items of the listing.
>> However, I'm wondering if such a design would play nicely with OCaml
>> data structures like Hashtbl? Any idea how to stop and restart an
>> iter-like operation on an OCaml Hashtbl.t?
>> Any idea of a better design?
> You may want to look at ExtLib's Enum concept.

Thanks for the pointer. I'll look at it.

> The standard way to fetch things from a database
> is that your 'select' request returns a cursor object;
> that is, an iterator. So you have operations:
> let it = get_selection <description> in
> try while true do
>   let data = get_data it in
>   <do something with data>
>   advance it
> done with End_of_data ->
> close it

Yes, it looks more OCamlish. :)

Regarding the network side, I've asked the advice of some
colleagues. They advise me to use a simple index-based iterator (so
similar to your proposal, but where the iterator is explicit).

Needed network API (as seen from client side) :

 - num_elements = get_number_of_elements(): returns the maximum number
   of elements;

 - get_elements_range(index_base, size): returns /size/ elements,
   starting at /index_base/.

the client do a:

 i := 0
 while !i < get_number_of_elements() do
     el_list = get_elements_range(i, 10)
     <process el_list>
     i := i + 10

The advantages of this approach:

 - all the context is in the client, the server does not need to
   maintain state per client;

 - the degenerated case where the client can only fetch one element at a
   time is included (size = 1);

 - this approach is compatible with indexes as seen in real SQL

So I'll use this API, except if somebody has a better proposal (with a
rationale ;).

It remains to code all of that. :)

 David Mentré <address@hidden>

reply via email to

[Prev in Thread] Current Thread [Next in Thread]