Common CPP Classes for EventHandler Classes

[Mike Papper]

I have been looking at Common CPP for use in scaling our distributed system.
I want to use what is termed an "EventHandler" (from the SEDA project at
http://www.eecs.harvard.edu/~mdw/proj/seda/ ). The event handler is a Queue
and some number of threads that process items from the queue. It is used as a
component for event handling/message passing in a server.

I would like to create 2 types of "event handlers" and was wondering
someone with CommonCPP experience could give me feedback on what CPP classes
would be appropriate to combine together?

1) Message Processing Handler: input queue of Message objects and a pool of
threads that process the messages. Threads wait on the queue and when a queue
item is added, one thread wakes up and processes the message. Then the thread
goes back to waiting on the queue.

I would think this maps to the Buffer class (as described in the Threads.h
file). I am unclear as to how to subclass this and how to tie in some
pre-allocated threads.

Also, when we shutdown the process, can we easily tell these pooled threads
to stop and go away for a cleaner shutdown? I.e., if we join on these threads
but they are blocked on a semaphore, they wont join.

2) Socket Class: we have 1500 HTTP connections to this server. We will have a
number of instances of the Socket class to read data (but not to write data)
from these connections. We want to use a small number of threads to handle
the large number (1500) socket instances. And we want these small number of
threads to read efficiently.

Possible solutions for the Socket Class Event Handler:

1) I am looking at using non-blocking I/O and simply running down the list of
sockets non by one- can this be done with the Socket class?

2) select or poll: I have heard that using unix select or poll with 1500
items is bad (the list is too long). So, I was going to break it into a list 
of 200 items and a number of threads to reach the 1500. Will the Socket class 
allow me to do this or do I need to call poll or select directly 
myself?

3) I could have one thread per socket to read the data, process
all the messages, send data back and then keep reading (using keep-alive on
the socket).

4) Ideally a small number of threads that collectivly wait on the large
number of sockets for data to read. Is this concept in any CommonCPP classes? 
If not, how would I add such a structure?

Any ideas on how to implement any of these (especially with CommonCPP
classes) would be appreciated.

Are there any set of classes or methods for efficient reading of
a large number of socket connections with a relativly few number of threads?

--------------------------------  FYI   -------------------------
Our System:
---------------
A central machine communicates with 1500 "edge" machines. Each edge machine
is similar and connects to the central machine via HTTP, sends some "message"
data and waits for "message" data in the HTTP reply. Then the HTTP connection
is finished and the edge machine immediatly starts a new HTTP connection.

We use HTTP from edge to center to go through firewalls. We have the edge
machine always connecting to the central machine because we want to send
messages from the center to the edge as well as edge to center (i.e., since
the edge must initiate the socket connection, it must connect to the central
machine to receive any messages).

The central machine processes messages which does things like update the
status information about an edge, possibly update some information in a SQL
database, take any queued messages for this edge and send in the HTTP reply.

Platform:
-----------
We are using red hat 9 which has the "new" better thread support from the 2.6
kernel backported into it. C++.  We have access to multi-cpu boxes with lots
of RAM - well 4 Gb anyway.I am wondering if anyone has experience using many
threads on this new red hat 9 platform?

--
Mike Papper
Fantastic
address@hidden
650-356-2113