[Chicken-hackers] A couple of questions

[Jörg F . Wittenberger]

# Askemos – What is it?

I keep receiving positive comments to Askemos/BALL-related code posted
to Chicken lists. (Some examples below.) However people seem almost
scared to use it for their own good.

This code base became what I'd call a swiss-army knife to **simplify**
tasks like backup, version control, content management and
(web)development. -- I'm afraid there's a prevalent documentation
issue. Given that we're talking about quite some man-years of
development and testing by our team, the scaring appears less a
surprise.

To those even remotely interested: *please* bomb me with your
questions, requirements and interests (private replies - don't spam
the lists). I want to improve the docs, most needed info first.  I
want to serve your needs, not what I imagine your needs would be.

Find some features/keywords I guess you might love for inspiration
after the example comments. But: ask for anything you'd like to see
answered.

Thanks for your time.

/Jörg

## Comment Examples

On 25 Jan 2009, Alaric Snell-Pym wrote relating to proposed llrbtree I
wrote for chickens scheduler:

> Yes! I've been impressed with Jorg's work there, even though it
> won't directly affect me since I'm not doing anything highly
> multithreaded.


(see 
http://lists.nongnu.org/archive/html/chicken-users/2009-01/msg00104.html )

On Oct 17th 2011 David-Sarah Hopwood of tahoe-lafs fame wrote:

> Tahoe-LAFS is solving a much simpler problem than … Askemos


On 13 Jan 2013 Ian Grigg (iang.org) wrote:

> I'm actually unclear on what Askemos is…

> (later - found this
> http://ball.askemos.org/A26b5619be8d5e3348cca356acfc8efea/ )

> >  
> > http://ball.askemos.org/A26b5619be8d5e3348cca356acfc8efea/DevelopmentNetwork


> Wait -- I see an identifier. It leads to the same document on
> multiple sites. How does it do that? Are the sites sharing multiple
> copies, and keeping track in the background? Or accessing a common
> database? How do the sites keep track of changes?


(No common database. Active replication tolerating byzantine
faults. Updates while > 2/3rd of nodes are avail.)

On Jan 25 2013, Daniel Leslie wrote:

> I keep running across amazingly useful code snippets from you
> in the chicken-users list archive … examples: …

> http://lists.nongnu.org/archive/html/chicken-users/2010-12/msg00218.html


(A pthread-interface to run blocking C code without blocking the
Chicken thread. Plus a sqlite-interface using the former. The latter
uses sqlite's virtual file system to retrieve data blocks via
Chicken-Scheme again. This is the code used in turn for fault-tolerant
replication of the database(s) itself. BTW: Does anybody know another
(No~ or)SQL database with simple, peer-to-peer replication protecting
against byzantine faults?)

On Jan 21 2013, Matt Gushee wrote:

> I wanted to say that I share Daniel's thoughts about Askemos.  It
> seems like a really intriguing project, and I would like to learn
> more about it at some point. For the moment, my concerns are a bit
> more mundane: I want to create a CMS that is broadly similar to
> existing products such as Drupal, but with better performance and
> usability ... and that is usable in conventional web hosting
> environments.

Question: What means something to be "usable in a conventional web
hosting environment"?  No chance to run a binary to serve requests?

## Features - What's important to you?

If you have time enough to dig deeper: askemos.org is about
"design&requirements" ball.askemos.org about the actual software.

### Permission Control
* No omni-privileged root/administrator
* Unlimited many, structured capabilities (algebraic)
* Similar to http://www.iang.org/papers/ricardian_contract.html
   but applicable to running processes instead of static
   contracts
* legal opinion finds it superior to PKI. PDF (German):

   http://www.askemos.org/A0e80fdd97a7b6e7af87c5d294f39a96c


### License
* With RScheme: GPL (contracted over several years)
* With Chicken: BSD (tentative, some GPLed code snippets still not
 replaced by BSD equivalents) -- (and unfortunately still not running
 stable for days without restart as the RScheme version does)

### Networking
* SSL implementation as plugin (free the Scheme thread)
* encryption/auth plugin also to ease use of alternatives
   (GNUnet?)
* SOCKS4a for use with Tor
* Reverse use of HTTP to ease tunneling firewalls
* Updates DNS dynamically
* persistent connections forsake of speed

### MIME handling
* See: http://ball.askemos.org/?template=htmldoc PDF on-the-fly
   (date&time of retrieval at the bottom of the page)
* Defaults to XML for stored content (to avoid forcing a messy
   wiki-syntax onto users) even though applications may default
   to other syntax for edits. Compare:

   http://askemos.org/A74c3f2644819d5eb04fa18af4cb3972d?_v=spo&_id=1623
     - markdown, the default
    
http://askemos.org/A74c3f2644819d5eb04fa18af4cb3972d?_contenttype=text%2fxml&_v=spo&_id=1623
     - XML, the internal
    
http://askemos.org/A74c3f2644819d5eb04fa18af4cb3972d?_contenttype=text%2fwikipedia&_v=spo&_id=1623
     - a "failed" experiment: parse wikipedia-alike as LALR(1)

* For sure: does binary data.


### Application (Contract) Programming Languages
* Easy to change (plugin)
* Kept in metadata of corresponding object (field "Action"):

   http://askemos.org/A74c3f2644819d5eb04fa18af4cb3972d?xmlns=a
   above points to
   http://askemos.org/A214a8fe0e429281327b2492a5b658b2c

* Additional primitives easy to export from initialization; e.g.

   * the wiki (hosting ball.askemos.org ; only kept for backward
      compatibility) is compile code
   * WebDAV is compiled too

     Speaking of WebDAV: Not that I like it.  It's a horrible
     standard.  But: broadly supported IMHO.

     The implementation shares common data if possible.  (A hash-tree
     again.)

* Minimal Core API (similar to "basic Scheme in XML")

 http://askemos.org/index.html?_v=search&_id=1728

* So far: Scheme/DSSSL, XSLT (alike, several elements have equivalents
   with XPath replaced by Scheme/SXPath), SXML, XSQL and SQLite out
   of the box.

   Those mix with ease. A commented example:
   http://askemos.org/A556916925f56990cfa1ddd81b6ea1198?date=2010-11-01


### Safety
* See Languages example: SQL injection is hard. Needs use of
   depreciated and more complicated API in the first place.
* Application code runs in sandbox with resource limitations.
   (E.g., an endless loop will die with timeout.)


### Version Control
* Internally every is stored in Hash-trees (Merkle-Trees)
* Much like "git" in Scheme (at low level)
 -> just no version control policy **imposed**. That's delegated to
    user level code for customization.  (One size doesn't fit all.  I
    spent years at that for living)
* Example super-simple VC implementation: the "new" button here
   would create a snapshot. "r/o" and "publish" control what can
   be done to the snapshot. (Keep the identifier linked
   elsewhere to avoid garbage collection)

         http://askemos.org/A74c3f2644819d5eb04fa18af4cb3972d?_v=overview


### Administration
* To control you own cloud:

   * Open backdoor in cofiguration
   * connect to `` repl ``-port
   * set quorum
   * enter Scheme code

* One binary package + one runtime directory

* Tested: various R&D-projects in the past decade

* Efficiency

   * For updates: not yet. Update takes around half a second in
      network delay (WAN, two round byzantine agreement)
      independent of size of transaction
   * For reads: Hm; the more nodes you add, the better.


### Portability

* Most code R5RS; RScheme and Chicken special code for efficiency for
 low level stuff.
* Compatible up to SHA256 checksums among 32/64bit, Intel/ARM,
 RScheme/Chicken, Linux/FreeBSD constantly (con)tested in byzantine
 agreement.  (Don't ask what that means wrt. number formatting)
* Porting code among compiles doesn't make the code worse if you fix
 the cause, not the symptoms.


### Simple, Readable -- Debugging & Usage

* (Side)effects held back until commit stage
 -> intercept this to reason from (intermediate) source in formatted
    XML (input to the byzantine agreement) instead of effects in
    various results spread over your file system

* From 
http://lists.nongnu.org/archive/html/chicken-users/2013-01/msg00098.html

  When I need to change some source code, I mount the WebDAV share
  vial ssh tunnel use emacs, change and save. Then I test the changes
  with those apps configured to use the draft code.  Once it works I
  visit the snapshot page and push the "follow" button.  Now the
  change is effective at the websites.  Since this can still reveal
  bugs, both the first level and the second level directory have a
  button "go back one step".  (Similar the app behind askemos.org has
  a snapshot button.  That one allows to kinds of snapshots:
  immutable and mutable.)

  Once a year I keep a backup of those copies on disconnected media
  within a steel case.  Otherwise I rely on the self- healing via
  replication and fail-stop notice on lower bound.

  The normal thing is that for whatever reason there is almost always
  one of the replica not available.  Be the cable pulled of the wall
  by the cat.  Another hoster has a dog…


Thanks again for your time!  Anny and all comments appreciated!


/Jerry
....................