GNU ccScript "3" released for testing tonight

[David Sugar]

While not widely disclosed prior, a brand new version of the ccscript 
library has been completed this week and released tonight.  This new 
ccscript engine, libccscript3, is designed to finally overcome some of 
the major problems in past implementations of the scripting engine, as 
well as fixing issues that remained inadequately addressed in the 
testing branch and other versions.

First, there has been the question of accurate compilation.  This
problem exists in part because the language itself was never rigorously 
defined in the actual implementation because past releases of the script 
compiler did far too inadequate a job in enforcing compile time language 
checking.  While compile time checking has always been available in the 
ccscript engine, the check methods implemented were often vague and 
never updated to address the needs of enforcing syntax for specific 
commands.  ccScript3 includes complete and systematic rewrite of check 
methods and has compile time checking and enforcement of the entire 
scripting language definition.  All of the vague checking methods have 
been replaced with chk methods specific to each command.  This will 
eliminate script statements that are not valid but may seem to execute 
due to side effects, and which can crash the runtime engine.

Second, ccscript3 has a much simpler set of commands and more constant 
use of variable names and scoping rules than either prior library 
releases, or even the first attempt at simplification in the testing 
branch.  For example, the set command no longer has many odd and special 
.member effects or alternate syntaxes through keywords.  Instead, for 
example, set, init, and const now all share the same basic command 
forms, and these are enforced through compile time checks.

In addition, many commands that had odd .member directives and special 
side effects have been eliminated.  For example, the goto command no 
longer has the option to set variables as part of a goto.  Other odd and 
"overloaded" combinations of functionality held within a single command 
statement have also been eliminated. In some cases, where the same 
command was used in different ways based on .member or other things, a 
new command for the alternate functionality was instead added.  Hence, 
for example, "goto" now refers only to script labels (consistent with 
call, gosub and source), while a new "signal" command is used to invoke
a local ^handler or @xxx handler.  Again all these uses and restrictions 
are now enforced by compile time checking.

Third, many of the ideas for improving execution performance already 
tried in the testing branch have been expanded upon.  Fast branching for 
commands that jump scripts are used, and now include automatic execution 
of set, init, and const commands when not in trace mode without the need 
of explicit begin transaction blocks.  In addition, the model for script 
mutex blocking has been rewritten to eliminate unessisary recursive 
locking.  This also means much faster and simpler execution of symbol 
operations, as all mutex operations have been moved into the interp 
class through explicit public members, rather than appearing and getting 
recursivily invoked in low level operations in the symbol table class.

Many of the odd and overloaded command constructs came into existence 
because of the step execution model of ccscript.  The idea being one can 
use shortcuts by overloading key commands with multiple functions so 
different functions needed together can execute as a single step.  The 
improvements in transaction processing, and in automatic execution of 
symbol definition operations, make such overloading, already hard to 
document, completely unnecessary to improve performance.  This is also 
why the scripting language was finally able to be simplified and 
standardized in ccscript3.

Fourth, some key improvements from the testing branch were carried 
forward into ccscript3.  These include the use of script function 
definitions that can include default arguments.  Stack frames are 
automatically managed as in the testing branch, and no longer require 
explicit pop commands as was at times necessary in the base 
distribution.  The use of inherited event handlers has also been kept 
making it possible to write functions that operate as if they are 
command extensions much like the testing branch uses.

Fifth, the optional use of script "constructors", first found in the 
testing branch, have been expanded upon.  This feature eliminated the 
many of the odd preprocessor .directives found in Bayonne to initialize 
specific variables at compile time or during execution startup.  Instead 
of using an alternate preprocessor syntax, standard const and set 
statements can be used for this purpose.  As in the testing branch, if a 
program "main" directive is used, or a "module" is compiled, the 
non-label script commands contained at the start of the script are 
automatically executed to initialize variables to known values.  Unlike 
the testing branch, libccscript3 is now able to further verify and 
enforce (the testing branch only checked at runtime if the script step 
invoked states) which command statements will be allowed to initialize a 
new engine session, and these are tested at compile time.

The first real difference in ccscript3 over all past implementations is 
that it directly supports the use of threaded plugins.  These were only 
possible to build and test before entirely within Bayonne itself.  This 
means the entire system for processing databases in ccscript, for 
example, can exist and be used as a pure ccscript plugin without 
requiring Bayonne.

Being able to test plugins like database support outside of Bayonne 
means they will be much easier to code, test, and debug.  Indeed, where 
we used the testscript shell in the testing branch to support "test 
case" scripts to verify and validate the scripting engine, we can now 
write "test cases" to verify the operation and correctness of database 
operations for the first time without needing to run them from Bayonne.

One feature not carried forward from the testing branch is some of the 
extensive new support for data manipulation.  ccscript was never 
conceived of as a rigorous data manipulation language, but rather 
principally for defining call control/call flow and as an extensible 
scripting engine shell, much like for example the bash shell does. 
Things that involve complex data manipulation should be carried out in 
plugins that are invoked through the scripting language.

However, some very useful new extensions were added.  One of them is the 
ability to access and even programatically manage the new version of the 
foreach "iterative" loop operator.  The foreach can expose it's index 
through an external variable, and ^event handlers can aquire and modify 
index marks.

Another important change is that arrays and stacks no longer have a 
limit of 255 entries.  All values for management of arrays and stacks is 
now a unsigned short (64k entries).   The code for this no longer 
depends on strange offset bytes in sym->data[x], as a  special Array 
header object is now imbedded, making the coding much simpler and more 
reliable.   For this reason, the new ccscript database class uses only 
one method of returning query results; to a ccscript symbol, array, or 
stack/fifo, and the now universal foreach command can be used to iterate 
multi-row results, eliminating the need for special for.data and 
for.array stuff from older ccscript versions.

Many of the most important improvements in libccscript3 are not those 
immediately visible.  Among these was a great simplification in the code 
for managing symbols, and the new methodology for handling "call by 
reference" function arguments, which were originally done as a hack in 
the testing branch.  This new symbol by reference support replaces both 
the call by reference code in the testing branch and even the old 
ccscript alias command.  Other important changes include the 
partitioning of source files and allocation of member functions so it is 
much easier to examine and debug the libccscript3 code itself.  Also, 
the use of special ".properties" has been completely eliminated, and 
this has made symbol lookup faster since it does not have to occur 
twice.  In it's place, a special #var symbol function has been added.

Another key change is the use of sideway class inheritance.  This means 
that future Bayonne drivers will not have huge classes that inherit the 
entire script engine all the way down to the driver.  It means plugin 
extensions can be written that directly bind to the command hash table 
and have full compile time check functions, rather than using a separate 
ScriptModule class and separate linked lists for handling plugin based 
extensions.  There are now also a core set of classes that purely 
represent the script execution environment alone without any actual 
commands, language structure, check methods, or script compiler, all of 
which are separated out of the ccscript base into sideway classes linked 
together by the method pointer list that is loaded into the command 
image class.

The first release is available from ftp.gnu.org under gnu/ccscript as 
libccscript3.

dyfet.vcf
Description: Vcard