[Gzz-commits] manuscripts/pointers article.rst

[Benja Fallenstein]

CVSROOT:        /cvsroot/gzz
Module name:    manuscripts
Branch:         
Changes by:     Benja Fallenstein <address@hidden>      03/11/03 18:44:07

Modified files:
        pointers       : article.rst 

Log message:
        start working on block model section

CVSWeb URLs:
http://savannah.gnu.org/cgi-bin/viewcvs/gzz/manuscripts/pointers/article.rst.diff?tr1=1.105&tr2=1.106&r1=text&r2=text

Patches:
Index: manuscripts/pointers/article.rst
diff -u manuscripts/pointers/article.rst:1.105 
manuscripts/pointers/article.rst:1.106
--- manuscripts/pointers/article.rst:1.105      Mon Nov  3 16:22:06 2003
+++ manuscripts/pointers/article.rst    Mon Nov  3 18:44:06 2003
@@ -302,41 +302,59 @@
 Primitive block storage abstraction
 ===================================
 
-In this section, we introduce a simple data model that
-
-
-explicitly state the capabilities
-that our pointer scheme requires from the underlying system.
-
-
-Add permanent block
-    Given a bit string and a MIME type,
-    adds the block into
-    the system and return a *permanent id* for
-    that exact bitstring,
-    for example a cryptographic hash of the data.
-
-Get the permanent data for a given id
-    Given a permanent id, locate and return the bitstring
-    stored with that permanent id, or inform the caller
-    that the data was not currently available.
-
-Global reverse indices
-    The ability to define (on a system-wide basis)
-    partial functions of blocks (of the data in the bitstrings)
-    and to query for the ids
-    (and thus the data)
-    of the blocks with the given values for the functions.
-
-    For scalability, the ability to also define another 
-    function, which takes a set of blocks and determines which
-    of the blocks are "relevant" locally on each peer can yield
-    great improvements.
-
-Managing where the blocks are stored is excluded
-from the scope of this abstraction. The versioning system
-must work as best it can even if only a subset of the
-blocks is available. 
+In this section, we introduce the Storm [#]_ model, 
+a simple but powerful data model that can represent
+all data on the P2P Web and that maps well to
+the functionality of current P2P filesharing systems.
+
+.. [#] For STORage Module. Storm is the name of our
+   Free Software implementation, discussed in
+   Section XXX.
+
+The Storm model is comprised of the following components:
+
+Blocks, identified by hash
+    All data is stored in *blocks*, which are file-like,
+    immutable byte sequences. Blocks have global identifiers
+    composed of a cryptographic hash of their content
+    plus a MIME content type. (The MIME type is the minimal amount
+    of metadata necessary in addition to the content 
+    for showing a block in a Web browser.)
+
+Pools
+    A *pool* is a collection of blocks. The data published
+    by a particular P2P Web server, the data downloaded
+    by a client, the data in a client's cache, and all the
+    data published on the whole Web can all be seen as a pool.
+    In an API, a pool provides the functionality to
+    fetch a block by its global id; local pools also allow
+    new blocks to be added.
+
+Reverse indices
+    Reverse indices are application-specific plug-ins
+    which index blocks by properties of their content
+    rather than by hash. Reverse indices examine each
+    block in a local pool and return hashtable keys
+    under which this block should be found. 
+
+    For example, an index of Ogg Vorbis audio blocks running 
+    on an artist's Web server could return each word
+    occuring in the album and title fields of each song.
+    The server would publish the resulting mappings
+    from words to block ids on the P2P network.
+
+An index can also return additional information
+to be placed in the hashtable along with the
+block id. In the example, this could be the complete
+metadata of the song, so that a client does not need
+to download the complete audio file before being able
+to show its artist, title etc. to its user.
+
+The pointer functionality is trivial to implement
+*on top of* this basic model. A pointer record
+is simply a special kind of block. A pointer record index
+is used to publish mappings from pointer ids
+to the block ids of pointer records.
 
 The benefits of starting from this basic model are numerous:
 
@@ -347,23 +365,6 @@
 
 - robust - all state of the system is in the blocks
 
-In this framework, we shall define a system of *pointers*:
-some types of identifiers that refer to a block but whose
-referent may be changed. In the following Sections, we show
-how a reasonably efficient system can be constructed
-using only the primitive datablock model given above.
-
-..  - Basic model of underlying system: Only blocks
-    - Benefits of that, in a nutshell
-    - Reverse indexing (nicely provided through structured overlays), 
-      *define* reverse indexing
-    - Def.: Pointers are abstract references that *point*
-      to different blocks (versions) over time
-    - Storm is designed to be modular; different routing algorithms
-      can be used with Storm for locating data with loc.ind. ids (
-      (e.g., PAST and CFS are tightly binded to their routing
-      algorithm)
-
 
 Applications
 ============
@@ -424,7 +425,9 @@
 pointer records could be used to implement
 a p2p version of software upgrades
 such as Debian's ``apt`` or Windows update,
-allowing large bandwidth savings.
+allowing for large bandwidth savings because
+updates could be downloaded from the closest
+host that has a copy.
 
 An important advantage of pointer records, due
 to the simple underlying block abstraction, is