[Bug-gnubg] Multiprocessing

[Olivier Baur]

Hi all!

At last! I've managed time (and nerves) to finish what I started a 
month ago -- multiprocessing now runs nicely with GTK!
There were a few thread-related undocumented caveats in GTK... And a 
few hard-to-spot deadlocks... I guess I'll have to drop a few lines 
into the GTK FAQ after what I've been going through ;-)

I've commited the changes to the CVS, so it's available for compilation 
in "branch-multi".

I've added the following GUI elements (maybe they're not properly 
placed, your opinion welcome): in the Edit menu, under the "Enter 
command..." item, I've added a submenu called "Procesing Units" that 
offers to bring up three different windows: "Master Mode Control" 
(modeless), "Slave Mode Control" (modeless), and "Options" (modal). 
Maybe I should move them to the Settings and Windows menus?

The Slave Mode Control window is very straightforward: click "Start" to 
put gnubg into slave mode, "Stop" to revert it to master mode; the 
button "Options" is a shortcut to the Options window.

The Master Mode Control window displays the list of all installed 
processing units (either local, ie dedicated to processors, in case 
you're running on a multi-processor machine, or remote, ie connections 
to slave gnubg's), and a bunch of buttons:
- "Add local" adds a local processing unit and starts it (should not be 
necessary, unless gnubg doesn't detect the correct number of installed 
processors at startup)
- "Add remote" prompts you for the address of a slave, adds it as a 
remote processing unit and starts it;
- "Remove" removes the selected processing units;
- "Start" starts the selected processing units which are currently 
stopped; in case of a remote procunit, establishes the TCP connection 
with the slave (which goes to "Idle" state);
- "Stop" stops the selected processing units which are currently not 
stopped (ie ready or busy); in case of a remote processing unit, closes 
the connection with the slave (which goes to "Wainting for connection" 
state)
- "Set Queue Size" sets the queue size of the selected remote 
processing units; eg, if you set the queue size of a rpu to 4, tasks 
will be sent to the corresponding slave in jobs of 4 tasks, reducing 
the importance of the network operation overhead compared to real 
backgammon processing; the default queue size of a rpu is the number of 
processing units this slave sports.
- "Show Stats" does not work yet
- "Options" is a shortcut to the Options window

The Options window allows you to set up parameters for both master and 
slave modes
* Master Options:
- "Listen for slave availability notifications": if checked, each slave 
that notifies itself to this master gnubg will be automatically added 
as a remote processing unit (even while a rollout is already being 
performed)
- "Minimum RPU Que Size": does not work yet
* Slave Options:
- "No notification": slave will not notify any master; "Notify all 
masters on local network": slave will broadcast availability 
notifications to all masters reachable on the local network; "Notify 
specific master": slave will notify its availability to specified 
master gnubg;
* Both:
- "Notify/Listen on TCP port": specify on which TCP port slave/master 
gnubg should send/listen to availability notifications.

I'm currently writing all this down in a more detailed way in a texi 
file to be included in a gnubg.texi chapter/section (I was thinking of 
Chapter Settings, Section Processing?)


CAVEATS
- when performing a rollout, the "Rollout Progress Window" is not modal 
anymore, so you can interact with the Master Mode Control window (to 
start/stop procunits, maybe add new remote processing units while the 
rollout is being performed); do not try to use the regular gnubg menus 
(File, Edit, Analyse, etc) until you have dismissed the rollout dialog, 
as gnubg is not reentrant (in its non-mutliprocessing version, gnubg 
locks access to other windows during rollouts to prevent any unwanted 
reentrancy)
- do not try to run a single gnubg process both in master and slave 
mode; ie, do not create remote processing units, and then turn your 
gnubg into a slave for another gnubg master [the processing unit "task 
engine" is not reentrant -- yet]
- processing units settings can't be saved to file; however you can set 
up a bunch of remote hosts on a master at startup time using a usual 
command line script (using "pu add remote ..." or "set pu remote 
notification listen enabled on" commands)
- caution: use the same neural nets and match equity tables on all 
co-operating gnubg's;
- some errors (actually very few, mostly uncommon network errors) may 
be reported to stderr (by default, the terminal window from which gnubg 
is run) rather than into an error dialog, due to some issues involved 
in running GTK dialogs in threads;
- some threads might unnecessary take CPU power while doing nothing 
(eg, slave gnubg in Waiting for connection state) [this is due to some 
bad programming techniques I have used to get the whole thing running 
with GTK; the issue can be solved by merely moving these pieces of code 
to different threads of execution and have then block normally onto 
mutexes rather than poll semaphores AND run the gtk_main() loop at the 
same time -- but I'll want to be sure that everything else works before 
going to that]
- I've not been able to test wether data were correctly exchanged 
between big and small endian platforms


TO DO
- fix above issues
- add timeouts to rpus
- add processing analysis and n-ply evals
- and many other details...

I'm back to work on this now, so the CVS might get updated on a daily 
basis.

I hope someone can test this on a larger scale than I can -- Ian?


-- Olivier