Re: [Demexp-dev] How to get through the network an unknown amount of data?

[skaller]

On Thu, 2004-09-09 at 05:51, David MENTRE wrote:
> Hello,
> 
> At several point of the demexp network protocol, I need to get an a
> priori unknown amount of data. For example, the list of participants or
> the list of Questions in the database.
> 
> It would be stupid to get all the data in one (potentially very big)
> message. So I'm wondering what would be the best API to do that.
> 
> One obvious answer would be to use an API similar to Unix API to list
> files in a directory:
> 
>  - a function start_listing() to start the listing;
> 
>  - a function continue_listing() to have the let's say next 10
>    items of the listing.
> 
> However, I'm wondering if such a design would play nicely with OCaml
> data structures like Hashtbl? Any idea how to stop and restart an
> iter-like operation on an OCaml Hashtbl.t?
> 
> Any idea of a better design?

You may want to look at ExtLib's Enum concept.

The standard way to fetch things from a database
is that your 'select' request returns a cursor object;
that is, an iterator. So you have operations:

let it = get_selection <description> in
try while true do
  let data = get_data it in
  <do something with data>
  advance it
done with End_of_data ->
close it

This is very similar to the usual file handle idea.
And just like file handles, there will typically
be some buffering on both the client and server
side of the iterator.

If you have a GUI displaying a 'window' on
this data, you may need to make the cursor
thing bidirectional. The effect would be that
the client program only needs to explicitly
buffer a GUI page of entries.

-- 
John Skaller, mailto:address@hidden
voice: 061-2-9660-0850, 
snail: PO BOX 401 Glebe NSW 2037 Australia
Checkout the Felix programming language http://felix.sf.net