[Date Prev][Date Next][Thread Prev][Thread Next][Date Index][Thread Index]
[GNUnet-SVN] [gnunet-texinfo] branch master updated: Makefile: PHONY for
|
From: |
gnunet |
|
Subject: |
[GNUnet-SVN] [gnunet-texinfo] branch master updated: Makefile: PHONY for clean. TODO: document some tasks developer.texi: multitable fixes, batch of settitle -> node changes gnunet.texi: New file in progress version.texi: Should be created by 'make' later on. This is a static copy. |
|
Date: |
Tue, 21 Mar 2017 17:26:43 +0100 |
This is an automated email from the git hooks/post-receive script.
ng0 pushed a commit to branch master
in repository gnunet-texinfo.
The following commit(s) were added to refs/heads/master by this push:
new b78b182 Makefile: PHONY for clean. TODO: document some tasks
developer.texi: multitable fixes, batch of settitle -> node changes
gnunet.texi: New file in progress version.texi: Should be created by 'make'
later on. This is a static copy.
b78b182 is described below
commit b78b1825cd54ad5db5ba666864a7c2b8c1921453
Author: ng0 <address@hidden>
AuthorDate: Fri Feb 17 16:58:07 2017 +0000
Makefile: PHONY for clean.
TODO: document some tasks
developer.texi: multitable fixes, batch of settitle -> node changes
gnunet.texi: New file in progress
version.texi: Should be created by 'make' later on. This is a static
copy.
---
Makefile | 12 +-
TODO | 6 +
developer.texi | 2463 +++++++++++++++-----------------------------------------
gnunet.texi | 76 ++
version.texi | 4 +-
5 files changed, 733 insertions(+), 1828 deletions(-)
diff --git a/Makefile b/Makefile
index fec0841..3da058f 100644
--- a/Makefile
+++ b/Makefile
@@ -1,10 +1,6 @@
-clean:
- rm *.aux
- rm *.log
- rm *.pdf
- rm *.cp
- rm *.cps
- rm *.toc
-
pdf:
texi2pdf --quiet *.texi
+
+.PHONY: clean
+clean:
+ rm *.aux *.log *.toc *.pdf *.cp *.cps
diff --git a/TODO b/TODO
new file mode 100644
index 0000000..0c89712
--- /dev/null
+++ b/TODO
@@ -0,0 +1,6 @@
+# CHAPTERS
+[] All files: move @settitle's to @node's
+[] All files: move @node's then to @node with specific sub-@'s
+
+# UNION
+[] Move all files -> gnunet.texi
diff --git a/developer.texi b/developer.texi
index ba5d920..224bc7f 100644
--- a/developer.texi
+++ b/developer.texi
@@ -1,9 +1,12 @@
-\input texinfo @c -*-texinfo-*-
+\input texinfo
address@hidden -*-texinfo-*-
@c %**start of header
@setfilename developer
@settitle Developer Handbook
@c %**end of header
address@hidden version.texi
+
@copying
Copyright @copyright{} 2017 ng0
@@ -311,22 +314,19 @@ Edition @value{EDITION} @*
This book is intended to be an introduction for programmers that want to
extend the GNUnet framework. GNUnet is more than a simple peer-to-peer
application. For developers, GNUnet is:
+
@itemize @bullet
@item Free software under the GNU General Public License, with a community
that believes in the GNU philosophy
-
@item
A set of standards, including coding conventions and architectural rules
-
@item
A set of layered protocols, both specifying the communication between peers as
well as the communication between components of a single peer.
-
@item
A set of libraries with well-defined APIs suitable for writing extensions
@end itemize
-
In particular, the architecture specifies that a peer consists of many
processes communicating via protocols. Processes can be written in almost
any language. C and Java APIs exist for accessing existing services and for
@@ -364,8 +364,8 @@ them.
New developers can have a look a the GNUnet tutorials for C and java available
in the src/ directory of the repository or under the following links:
address@hidden
address@hidden
+
address@hidden @bullet
@item GNUnet C tutorial
@item GNUnet Java tutorial
@end itemize
@@ -378,40 +378,33 @@ you feel you need access, you should contact
@uref{http://grothoff.org/christian/, Christian Grothoff}, GNUnet's maintainer.
The public subsystems on the GNUnet server that help developers are:
address@hidden @bullet
address@hidden @bullet
@item The Version control system keeps our code and enables distributed
development. Only developers with write access can commit code, everyone else
is encouraged to submit patches to the
@uref{http://mail.gnu.org/mailman/listinfo/gnunet-developers, developer
mailinglist}.
-
@item The GNUnet bugtracking system is used to track feature requests, open bug
reports and their resolutions. Anyone can report bugs, only developers can
claim to have fixed them.
-
@item A buildbot is used to check GNUnet builds automatically on a range of
platforms. Builds are triggered automatically after 30 minutes of no changes to
Git.
-
@item The current quality of our automated test suite is assessed using Code
coverage analysis. This analysis is run daily; however the webpage is only
updated if all automated tests pass at that time. Testcases that improve our
code coverage are always welcome.
-
@item We try to automatically find bugs using a static analysis scan. This scan
is run daily; however the webpage is only updated if all automated tests pass
at the time. Note that not everything that is flagged by the analysis is a bug,
sometimes even good code can be marked as possibly problematic. Nevertheless,
developers are encouraged to at least be aware of all issues in their code that
are listed.
-
@item We use Gauger for automatic performance regression visualization. Details
on how to use Gauger are here.
-
@item We use @uref{http://junit.org/, junit} to automatically test gnunet-java.
Automatically generated, current reports on the test suite are here.
-
@item We use Cobertura to generate test coverage reports for gnunet-java.
Current reports on test coverage are here.
@end itemize
@@ -426,8 +419,8 @@ that this description also lists projects that are far from
complete, including
even those that have literally not a single line of code in them yet.
GNUnet sub-projects in order of likely relevance are currently:
address@hidden @asis
address@hidden @asis
@item svn/gnunet Core of the P2P framework, including file-sharing, VPN and
chat applications; this is what the developer handbook covers mostly
@item
@@ -451,11 +444,9 @@ GNUnet nodes on testbeds for research, development,
testing and evaluation
cocoa-based GNUnet GUI (dead?)
@end table
-
We are also working on various supporting libraries and tools:
address@hidden
address@hidden
address@hidden @asis
@item svn/Extractor/ GNU libextractor (meta data extraction)
@item
svn/libmicrohttpd/ GNU libmicrohttpd (embedded HTTP(S) server library)
@@ -467,7 +458,6 @@ automated debugging of distributed systems
accessing satellite connection quality reports
@end table
-
Finally, there are various external projects (see links for a list of those
that have a public website) which build on top of the GNUnet framework.
@@ -479,8 +469,7 @@ This section gives a brief overview of the GNUnet source
code. Specifically, we
sketch the function of each of the subdirectories in the @code{gnunet/src/}
directory. The order given is roughly bottom-up (in terms of the layers of the
system).
address@hidden
address@hidden
address@hidden @asis
@item util/ --- libgnunetutil Library with general utility functions, all
GNUnet binaries link against this library. Anything from memory allocation and
@@ -717,346 +706,78 @@ build on top of the APIs of the framework.
The following table summarizes our current view of the stability of the
respective protocols or APIs:
address@hidden @columnfractions 0.0263157894736842
-0.0263157894736842 0.0263157894736842 0.0263157894736842 0.0263157894736842
-0.0263157894736842 0.0263157894736842 0.0263157894736842 0.0263157894736842
-0.0263157894736842 0.0263157894736842 0.0263157894736842 0.0263157894736842
-0.0263157894736842 0.0263157894736842 0.0263157894736842 0.0263157894736842
-0.0263157894736842 0.0263157894736842 0.0263157894736842 0.0263157894736842
-0.0263157894736842 0.0263157894736842 0.0263157894736842 0.0263157894736842
-0.0263157894736842 0.0263157894736842 0.0263157894736842 0.0263157894736842
-0.0263157894736842 0.0263157894736842 0.0263157894736842 0.0263157894736842
-0.0263157894736842 0.0263157894736842 0.0263157894736842 0.0263157894736842
-0.0263157894736842
-
address@hidden Subsystem
-
address@hidden P2P
-
address@hidden IPC
-
address@hidden C API
-
address@hidden util
-
address@hidden n/a
-
address@hidden n/a
-
address@hidden stable
-
address@hidden arm
-
address@hidden n/a
-
address@hidden stable
-
address@hidden stable
-
address@hidden ats
-
address@hidden n/a
-
address@hidden unstable
-
address@hidden testing
-
address@hidden block
-
address@hidden n/a
-
address@hidden n/a
-
address@hidden stable
-
address@hidden cadet
-
address@hidden testing
-
address@hidden testing
-
address@hidden testing
-
address@hidden consensus
-
address@hidden experimental
-
address@hidden experimental
-
address@hidden experimental
-
address@hidden core
-
address@hidden stable
-
address@hidden stable
-
address@hidden stable
-
address@hidden datacache
-
address@hidden n/a
-
address@hidden n/a
-
address@hidden stable
-
address@hidden datastore
-
address@hidden n/a
-
address@hidden stable
-
address@hidden stable
-
address@hidden dht
-
address@hidden stable
-
address@hidden stable
-
address@hidden stable
-
address@hidden dns
-
address@hidden stable
-
address@hidden stable
-
address@hidden stable
-
address@hidden dv
-
address@hidden testing
-
address@hidden testing
-
address@hidden n/a
-
address@hidden exit
-
address@hidden testing
-
address@hidden n/a
-
address@hidden n/a
-
address@hidden fragmentation
-
address@hidden stable
-
address@hidden n/a
-
address@hidden stable
-
address@hidden fs
-
address@hidden stable
-
address@hidden stable
-
address@hidden stable
-
address@hidden gns
-
address@hidden stable
-
address@hidden stable
-
address@hidden stable
-
address@hidden hello
-
address@hidden n/a
-
address@hidden n/a
-
address@hidden testing
-
address@hidden hostlist
-
address@hidden stable
-
address@hidden stable
-
address@hidden n/a
-
address@hidden identity
-
address@hidden stable
-
address@hidden stable
-
address@hidden n/a
-
address@hidden multicast
-
address@hidden experimental
-
address@hidden experimental
-
address@hidden experimental
-
address@hidden mysql
-
address@hidden stable
-
address@hidden n/a
-
address@hidden stable
-
address@hidden namestore
-
address@hidden n/a
-
address@hidden stable
-
address@hidden stable
-
address@hidden nat
-
address@hidden n/a
-
address@hidden n/a
-
address@hidden stable
-
address@hidden nse
-
address@hidden stable
-
address@hidden stable
-
address@hidden stable
-
address@hidden peerinfo
-
address@hidden n/a
-
address@hidden stable
-
address@hidden stable
-
address@hidden psyc
-
address@hidden experimental
-
address@hidden experimental
-
address@hidden experimental
-
address@hidden pt
-
address@hidden n/a
-
address@hidden n/a
-
address@hidden n/a
-
address@hidden regex
-
address@hidden stable
-
address@hidden stable
-
address@hidden stable
-
address@hidden revocation
-
address@hidden stable
-
address@hidden stable
-
address@hidden stable
-
address@hidden social
-
address@hidden experimental
-
address@hidden experimental
-
address@hidden experimental
-
address@hidden statistics
-
address@hidden n/a
-
address@hidden stable
-
address@hidden stable
-
address@hidden testbed
-
address@hidden n/a
-
address@hidden testing
-
address@hidden testing
-
address@hidden testing
-
address@hidden n/a
-
address@hidden n/a
-
address@hidden testing
-
address@hidden topology
-
address@hidden n/a
-
address@hidden n/a
-
address@hidden n/a
-
address@hidden transport
-
address@hidden stable
-
address@hidden stable
-
address@hidden stable
-
address@hidden tun
-
address@hidden n/a
-
address@hidden n/a
-
address@hidden stable
-
address@hidden vpn
-
address@hidden testing
-
address@hidden n/a
-
address@hidden n/a
address@hidden @columnfractions .20 .20 .20 .20
address@hidden Subsystem @tab P2P @tab IPC @tab C API
address@hidden util @tab n/a @tab n/a @tab stable
address@hidden arm @tab n/a @tab stable @tab stable
address@hidden ats @tab n/a @tab unstable @tab testing
address@hidden block @tab n/a @tab n/a @tab stable
address@hidden cadet @tab testing @tab testing @tab testing
address@hidden consensus @tab experimental @tab experimental @tab experimental
address@hidden core @tab stable @tab stable @tab stable
address@hidden datacache @tab n/a @tab n/a @tab stable
address@hidden datastore @tab n/a @tab stable @tab stable
address@hidden dht @tab stable @tab stable @tab stable
address@hidden dns @tab stable @tab stable @tab stable
address@hidden dv @tab testing @tab testing @tab n/a
address@hidden exit @tab testing @tab n/a @tab n/a
address@hidden fragmentation @tab stable @tab n/a @tab stable
address@hidden fs @tab stable @tab stable @tab stable
address@hidden gns @tab stable @tab stable @tab stable
address@hidden hello @tab n/a @tab n/a @tab testing
address@hidden hostlist @tab stable @tab stable @tab n/a
address@hidden identity @tab stable @tab stable @tab n/a
address@hidden multicast @tab experimental @tab experimental @tab experimental
address@hidden mysql @tab stable @tab n/a @tab stable
address@hidden namestore @tab n/a @tab stable @tab stable
address@hidden nat @tab n/a @tab n/a @tab stable
address@hidden nse @tab stable @tab stable @tab stable
address@hidden peerinfo @tab n/a @tab stable @tab stable
address@hidden psyc @tab experimental @tab experimental @tab experimental
address@hidden pt @tab n/a @tab n/a @tab n/a
address@hidden regex @tab stable @tab stable @tab stable
address@hidden revocation @tab stable @tab stable @tab stable
address@hidden social @tab experimental @tab experimental @tab experimental
address@hidden statistics @tab n/a @tab stable @tab stable
address@hidden testbed @tab n/a @tab testing @tab testing
address@hidden testing @tab n/a @tab n/a @tab testing
address@hidden topology @tab n/a @tab n/a @tab n/a
address@hidden transport @tab stable @tab stable @tab stable
address@hidden tun @tab n/a @tab n/a @tab stable
address@hidden vpn @tab testing @tab n/a @tab n/a
@end multitable
-
Here is a rough explanation of the values:
address@hidden @asis
address@hidden stable no incompatible changes are planned at this time; for
IPC/APIs, if
address@hidden @samp
address@hidden stable
+No incompatible changes are planned at this time; for IPC/APIs, if
there are incompatible changes, they will be minor and might only require
minimal changes to existing code; for P2P, changes will be avoided if at all
possible for the 0.10.x-series
address@hidden testing no incompatible changes are
+
address@hidden testing
+No incompatible changes are
planned at this time, but the code is still known to be in flux; so while we
have no concrete plans, our expectation is that there will still be minor
modifications; for P2P, changes will likely be extensions that should not break
existing code
address@hidden unstable changes are planned and will happen; however, they
+
address@hidden unstable
+Changes are planned and will happen; however, they
will not be totally radical and the result should still resemble what is there
now; nevertheless, anticipated changes will break protocol/API compatibility
address@hidden experimental changes are planned and the result may look nothing
like
+
address@hidden experimental
+Changes are planned and the result may look nothing like
what the API/protocol looks like today
address@hidden unknown someone should think about
-where this subsystem is headed
address@hidden n/a this subsystem does not have an
-API/IPC-protocol/P2P-protocol
+
address@hidden unknown
+Someone should think about where this subsystem headed
+
address@hidden n/a
+This subsystem does not have an API/IPC-protocol/P2P-protocol
@end table
@c ***************************************************************************
@@ -3717,61 +3438,14 @@ message i+1 after the i-th message was received. All
messages were sent over
the same connection and the time to establish the connection was not taken
into account since this overhead is miniscule in practice --- as long as a
connection is used for a significant number of messages.
address@hidden
address@hidden 0.166666666666667 0.166666666666667 0.166666666666667
-0.166666666666667 0.166666666666667 0.166666666666667
-
address@hidden Transport
-
address@hidden UDP
-
address@hidden TCP
-
address@hidden SMTP (Purdue sendmail)
-
address@hidden SMTP (RH 8.0)
-
address@hidden SMTP (SGL qmail)
-
address@hidden 11 bytes
-
address@hidden 31 ms
-
address@hidden 55 ms
-
address@hidden 781 s
-
address@hidden 77 s
-
address@hidden 24 s
-
address@hidden 407 bytes
-
address@hidden 37 ms
-
address@hidden 62 ms
-
address@hidden 789 s
-
address@hidden 78 s
-
address@hidden 25 s
-
address@hidden 1,221 bytes
-
address@hidden 46 ms
-
address@hidden 73 ms
-
address@hidden 804 s
-
address@hidden 78 s
-
address@hidden 25 s
address@hidden @columnfractions .20 .15 .15 .15 .15 .15
address@hidden Transport @tab UDP @tab TCP @tab SMTP (Purdue sendmail) @tab
SMTP (RH 8.0) @tab SMTP (SGL qmail)
address@hidden 11 bytes @tab 31 ms @tab 55 ms @tab 781 s @tab 77 s @tab 24 s
address@hidden 407 bytes @tab 37 ms @tab 62 ms @tab 789 s @tab 78 s @tab 25 s
address@hidden 1,221 bytes @tab 46 ms @tab 73 ms @tab 804 s @tab 78 s @tab 25 s
@end multitable
-
The benchmarks show that UDP and TCP are, as expected, both significantly
faster compared with any of the SMTP services. Among the SMTP implementations,
there can be significant differences depending on the SMTP configuration.
@@ -3998,14 +3672,15 @@ stack. For the communication with the other devices I
used the RFCOMM
protocol. Also I used the HCI protocol to gain some control over the device.
The helper binary takes a single argument (the name of the Bluetooth
interface) and is separated in two stages:
address@hidden address@hidden @bullet
+
address@hidden %** 'THE INITIALIZATION' should be in bigger letters or stand
out, not
address@hidden %** starting a new section?
@node THE INITIALIZATION
address@hidden %**end of header
address@hidden @bullet
address@hidden @bullet
@item first, it checks if we have root privilegies (@emph{Remember that we need
to have root privilegies in order to be able to bring the interface up if it is
-down or to change its state.} ).
+down or to change its state.}).
@item second, it verifies if the interface with the given name exists.
@@ -4017,7 +3692,6 @@ interface:}
On the @emph{open_device} method:
@itemize @bullet
-
@item creates a HCI socket used to send control events to the the device
@item searches for the device ID using the interface name
@item saves the device MAC address
@@ -4035,8 +3709,9 @@ to get the port on which this device is listening on)
@strong{If the interface is not a Bluetooth interface the helper exits with a
suitable error}
@end itemize
+
address@hidden %** Same as for @node entry above
@node THE LOOP
address@hidden %**end of header
The helper binary uses a list where it saves all the connected neighbour
devices (@emph{neighbours.devices}) and two buffers (@emph{write_pout} and
@@ -4044,26 +3719,20 @@ devices (@emph{neighbours.devices}) and two buffers
(@emph{write_pout} and
the device's MAC address in order to announce the peer presence to the
neighbours. Here are a short description of what happens in the main loop:
-
@itemize @bullet
-
-
@item Every time when it receives something from the STDIN it processes the
data and saves the message in the first buffer (@emph{write_pout}). When it has
something in the buffer, it gets the destination address from the buffer,
searches the destination address in the list (if there is no connection with
that device, it creates a new one and saves it to the list) and sends the
message.
-
@item Every time when it receives something on the listening socket it accepts
the connection and saves the socket on a list with the reading sockets.
-
@item Every time when it receives something from a reading socket it parses the
message, verifies the CRC and saves it in the @emph{write_std} buffer in order
to be sent later to the STDOUT.
@end itemize
-
So in the main loop we use the select function to wait until one of the file
descriptor saved in one of the two file descriptors sets used is ready to use.
The first set (@emph{rfds}) represents the reading set and it could contain the
@@ -4081,26 +3750,25 @@ server and searche the registered service for that
device.
@emph{You should be aware of the fact that if the device fails to connect to
another one when trying to send a message it will attempt one more time. If it
-fails again, then it skips the message.}@ @emph{Also you should know that the
+fails again, then it skips the message.}
address@hidden you should know that the
transport Bluetooth plugin has support for @strong{broadcast messages}.}
address@hidden Detailes about the broadcast implementation
address@hidden Details about the broadcast implementation
@c %**end of header
First I want to point out that the broadcast functionality for the CONTROL
messages is not implemented in a conventional way. Since the inquiry scan time
-is too big@ and it will take some time to send a message to all the
+is too big and it will take some time to send a message to all the
discoverable devices I decided to tackle the problem in a different way. Here
is how I did it:
address@hidden @bullet
address@hidden @bullet
@item If it is the first time when I have to broadcast a message I make an
inquiry scan and save all the devices' addresses to a vector.
-
@item After the inquiry scan ends I take the first address from the list and I
try to connect to it. If it fails, I try to connect to the next one. If it
succeeds, I save the socket to a list and send the message to the device.
-
@item When I have to broadcast another message, first I search on the list for
a new device which I'm not connected to. If there is no new device on the list
I go to the beginning of the list and send the message to the old devices.
@@ -4108,35 +3776,26 @@ After 5 cycles I make a new inquiry scan to check out
if there are new
discoverable devices and save them to the list. If there are no new
discoverable devices I reset the cycling counter and go again through the old
list and send messages to the devices saved in it.
address@hidden itemize
address@hidden :
address@hidden @bullet
address@hidden:
address@hidden @bullet
@item every time when I have a broadcast message I look up on the list for a
new device and send the message to it
-
@item if I reached the end of the list for 5 times and I'm connected to all the
devices from the list I make a new inquiry scan. @emph{The number of the list's
cycles after an inquiry scan could be increased by redefining the MAX_LOOPS
variable}
-
address@hidden when there are no new devices I send messages to the old ones.
@end
-itemize
-
-
address@hidden
address@hidden when there are no new devices I send messages to the old ones.
address@hidden itemize
Doing so, the broadcast control messages will reach the devices but with delay.
-
-
@emph{NOTICE:} When I have to send a message to a certain device first I check
on the broadcast list to see if we are connected to that device. If not we try
to connect to it and in case of success we save the address and the socket on
the list. If we are already connected to that device we simply use the socket.
address@hidden itemize
address@hidden itemize
-
@node Windows functionality
@c %**end of header
@@ -4155,18 +3814,15 @@ that you should have the latest updates (especially the
@emph{ws2bth} header).
Besides the fact that it uses the Windows Sockets, the Windows implemenation
follows the same principles as the Linux one:
address@hidden @bullet
address@hidden @bullet
@item
It has a initalization part where it initializes the Windows Sockets, creates a
RFCOMM socket which will be binded and switched to the listening mode and
registers a SDP service.
-
In the Microsoft Bluetooth API there are two ways to work with the SDP:
@itemize @bullet
-
@item an easy way which works with very simple service records
-
@item a hard way which is useful when you need to update or to delete the
record
@end itemize
@@ -4183,8 +3839,7 @@ I used the GNUNET_NETWORK library for functions like
@emph{accept},
@emph{bind}, @emph{connect} or @emph{select}. I decided to use the
GNUNET_NETWORK library because I also needed to interact with the STDIN and
STDOUT handles and on Windows the select function is only defined for sockets,
-and it will not work for arbitrary file handles. @end itemize
-
+and it will not work for arbitrary file handles.
Another difference between Linux and Windows implementation is that in Linux,
the Bluetooth address is represented in 48 bits while in Windows is
@@ -4198,16 +3853,13 @@ broadcast messages. When it receives a broadcast
message it will skip it.
@c %**end of header
@itemize @bullet
-
@item Implement the broadcast functionality on Windows @emph{(currently working
on)}
-
@item Implement a testcase for the helper :@ @emph{@ The testcase consists of a
program which emaluates the plugin and uses the helper. It will simulate
connections, disconnections and data transfers.@ }
@end itemize
-
If you have a new idea about a feature of the plugin or suggestions about how
I could improve the implementation you are welcome to comment or to contact
me.
@@ -4246,28 +3898,24 @@ communications between nodes in the GNUnet overlay
network. CORE builds on the
TRANSPORT subsystem which provides for the actual, insecure, unreliable
link-layer communication (for example, via UDP or WLAN), and then adds
fundamental security to the connections:
address@hidden @bullet
address@hidden @bullet
@item confidentiality with so-called perfect forward secrecy; we use
@uref{http://en.wikipedia.org/wiki/Elliptic_curve_Diffie%E2%80%93Hellman,
ECDHE} powered by @uref{http://cr.yp.to/ecdh.html, Curve25519} for the key
exchange and then use symmetric encryption, encrypting with both
@uref{http://en.wikipedia.org/wiki/Rijndael, AES-256} and
@uref{http://en.wikipedia.org/wiki/Twofish, Twofish}
-
@item @uref{http://en.wikipedia.org/wiki/Authentication, authentication} is
achieved by signing the ephemeral keys using @uref{http://ed25519.cr.yp.to/,
Ed25519}, a deterministic variant of @uref{http://en.wikipedia.org/wiki/ECDSA,
ECDSA}
-
@item integrity protection (using @uref{http://en.wikipedia.org/wiki/SHA-2,
SHA-512} to do @uref{http://en.wikipedia.org/wiki/Authenticated_encryption,
encrypt-then-MAC)}
-
@item @uref{http://en.wikipedia.org/wiki/Replay_attack, replay} protection
(using nonces, timestamps, challenge-response, message counters and ephemeral
keys)
-
@item liveness (keep-alive messages, timeout)
@end itemize
@@ -4403,8 +4051,8 @@ When a client connects to the CORE service, it first
sends a
message type values which are supported by the application. The options
bitmask specifies which events the client would like to be notified about. The
options include:
address@hidden @asis
address@hidden @asis
@item GNUNET_CORE_OPTION_NOTHING No notifications
@item
GNUNET_CORE_OPTION_STATUS_CHANGE Peers connecting and disconnecting
@@ -4420,12 +4068,12 @@ GNUNET_CORE_OPTION_HDR_OUTBOUND Just the
@code{MessageHeader} of all outbound
messages
@end table
-
Typical applications will only monitor for connection status changes.
The CORE service responds to the @code{InitMessage} with an
@code{InitReplyMessage} which contains the peer's identity. Afterwards, both
CORE and the client can send messages.
+
@node Notifications
@c %**end of header
@@ -4463,28 +4111,28 @@ for each request).
@node Creating the EphemeralKeyMessage
@c %**end of header
- When the CORE service starts, each peer creates a fresh ephemeral (ECC)
- public-private key pair and signs the corresponding @code{EphemeralKeyMessage}
- with its long-term key (which we usually call the peer's identity; the hash of
- the public long term key is what results in a @code{struct
- GNUNET_PeerIdentity} in all GNUnet APIs. The ephemeral key is ONLY used for an
- @uref{http://en.wikipedia.org/wiki/Elliptic_curve_Diffie%E2%80%93Hellman,
- ECDHE} exchange by the CORE service to establish symmetric session keys. A
- peer will use the same @code{EphemeralKeyMessage} for all peers for
- @code{REKEY_FREQUENCY}, which is usually 12 hours. After that time, it will
- create a fresh ephemeral key (forgetting the old one) and broadcast the new
- @code{EphemeralKeyMessage} to all connected peers, resulting in fresh
- symmetric session keys. Note that peers independently decide on when to
- discard ephemeral keys; it is not a protocol violation to discard keys more
- often. Ephemeral keys are also never stored to disk; restarting a peer will
- thus always create a fresh ephemeral key. The use of ephemeral keys is what
- provides @uref{http://en.wikipedia.org/wiki/Forward_secrecy, forward secrecy}.
-
- Just before transmission, the @code{EphemeralKeyMessage} is patched to reflect
- the current sender_status, which specifies the current state of the connection
- from the point of view of the sender. The possible values are:
address@hidden @asis
+When the CORE service starts, each peer creates a fresh ephemeral (ECC)
+public-private key pair and signs the corresponding @code{EphemeralKeyMessage}
+with its long-term key (which we usually call the peer's identity; the hash of
+the public long term key is what results in a @code{struct
+GNUNET_PeerIdentity} in all GNUnet APIs. The ephemeral key is ONLY used for an
address@hidden://en.wikipedia.org/wiki/Elliptic_curve_Diffie%E2%80%93Hellman,
+ECDHE} exchange by the CORE service to establish symmetric session keys. A
+peer will use the same @code{EphemeralKeyMessage} for all peers for
address@hidden, which is usually 12 hours. After that time, it will
+create a fresh ephemeral key (forgetting the old one) and broadcast the new
address@hidden to all connected peers, resulting in fresh
+symmetric session keys. Note that peers independently decide on when to
+discard ephemeral keys; it is not a protocol violation to discard keys more
+often. Ephemeral keys are also never stored to disk; restarting a peer will
+thus always create a fresh ephemeral key. The use of ephemeral keys is what
+provides @uref{http://en.wikipedia.org/wiki/Forward_secrecy, forward secrecy}.
+
+Just before transmission, the @code{EphemeralKeyMessage} is patched to reflect
+the current sender_status, which specifies the current state of the connection
+from the point of view of the sender. The possible values are:
address@hidden @asis
@item KX_STATE_DOWN Initial value, never used on the network
@item
KX_STATE_KEY_SENT We sent our ephemeral key, do not know the key of the other
@@ -4499,100 +4147,82 @@ keep-alives)
operation; the other peer has so far failed to confirm a working connection
using the new ephemeral key
@end table
address@hidden Establishing a connection
address@hidden %**end of header
-
address@hidden Top
-
-
-
- Peers begin their interaction by sending a @code{EphemeralKeyMessage} to the
- other peer once the TRANSPORT service notifies the CORE service about the
- connection. A peer receiving an @code{EphemeralKeyMessage} with a status
- indicating that the sender does not have the receiver's ephemeral key, the
- receiver's @code{EphemeralKeyMessage} is sent in response.@ Additionally, if
- the receiver has not yet confirmed the authenticity of the sender, it also
- sends an (encrypted)@code{PingMessage} with a challenge (and the identity of
- the target) to the other peer. Peers receiving a @code{PingMessage} respond
- with an (encrypted) @code{PongMessage} which includes the challenge. Peers
- receiving a @code{PongMessage} check the challenge, and if it matches set the
- connection to @code{KX_STATE_UP}.
address@hidden Encryption and Decryption
address@hidden %**end of header
-
address@hidden Top
-
-
- All functions related to the key exchange and encryption/decryption of
- messages can be found in @code{gnunet-service-core_kx.c} (except for the
- cryptographic primitives, which are in @code{util/crypto*.c}).@ Given the key
- material from ECDHE, a
- @uref{http://en.wikipedia.org/wiki/Key_derivation_function, Key derivation
- function} is used to derive two pairs of encryption and decryption keys for
- AES-256 and TwoFish, as well as initialization vectors and authentication keys
- (for @uref{http://en.wikipedia.org/wiki/HMAC, HMAC}). The HMAC is computed
- over the encrypted payload. Encrypted messages include an iv_seed and the HMAC
- in the header.
-
- Each encrypted message in the CORE service includes a sequence number and a
- timestamp in the encrypted payload. The CORE service remembers the largest
- observed sequence number and a bit-mask which represents which of the previous
- 32 sequence numbers were already used. Messages with sequence numbers lower
- than the largest observed sequence number minus 32 are discarded. Messages
- with a timestamp that is less than @code{REKEY_TOLERANCE} off (5 minutes) are
- also discarded. This of course means that system clocks need to be reasonably
- synchronized for peers to be able to communicate. Additionally, as the
- ephemeral key changes every 12h, a peer would not even be able to decrypt
- messages older than 12h.
address@hidden Type maps
address@hidden %**end of header
-
address@hidden Top
-
-
-
- Once an encrypted connection has been established, peers begin to exchange
- type maps. Type maps are used to allow the CORE service to determine which
- (encrypted) connections should be shown to which applications. A type map is
- an array of 65536 bits representing the different types of messages understood
- by applications using the CORE service. Each CORE service maintains this map,
- simply by setting the respective bit for each message type supported by any of
- the applications using the CORE service. Note that bits for message types
- embedded in higher-level protocols (such as MESH) will not be included in
- these type maps.
-
- Typically, the type map of a peer will be sparse. Thus, the CORE service
- attempts to compress its type map using @code{gzip}-style compression
- ("deflate") prior to transmission. However, if the compression fails to
- compact the map, the map may also be transmitted without compression
- (resulting in @code{GNUNET_MESSAGE_TYPE_CORE_COMPRESSED_TYPE_MAP} or
- @code{GNUNET_MESSAGE_TYPE_CORE_BINARY_TYPE_MAP} messages respectively). Upon
- receiving a type map, the respective CORE service notifies applications about
- the connection to the other peer if they support any message type indicated in
- the type map (or no message type at all). If the CORE service experience a
- connect or disconnect event from an application, it updates its type map
- (setting or unsetting the respective bits) and notifies its neighbours about
- the change. The CORE services of the neighbours then in turn generate connect
- and disconnect events for the peer that sent the type map for their respective
- applications. As CORE messages may be lost, the CORE service confirms
- receiving a type map by sending back a
- @code{GNUNET_MESSAGE_TYPE_CORE_CONFIRM_TYPE_MAP}. If such a confirmation (with
- the correct hash of the type map) is not received, the sender will retransmit
- the type map (with exponential back-off).
address@hidden @bullet
-
-
address@hidden
-
address@hidden itemize
address@hidden GNUnet's CADET subsystem
address@hidden Establishing a connection
address@hidden %**end of header
+
+Peers begin their interaction by sending a @code{EphemeralKeyMessage} to the
+other peer once the TRANSPORT service notifies the CORE service about the
+connection. A peer receiving an @code{EphemeralKeyMessage} with a status
+indicating that the sender does not have the receiver's ephemeral key, the
+receiver's @code{EphemeralKeyMessage} is sent in response.@ Additionally, if
+the receiver has not yet confirmed the authenticity of the sender, it also
+sends an (encrypted)@code{PingMessage} with a challenge (and the identity of
+the target) to the other peer. Peers receiving a @code{PingMessage} respond
+with an (encrypted) @code{PongMessage} which includes the challenge. Peers
+receiving a @code{PongMessage} check the challenge, and if it matches set the
+connection to @code{KX_STATE_UP}.
+
address@hidden Encryption and Decryption
address@hidden %**end of header
+
+All functions related to the key exchange and encryption/decryption of
+messages can be found in @code{gnunet-service-core_kx.c} (except for the
+cryptographic primitives, which are in @code{util/crypto*.c}).@ Given the key
+material from ECDHE, a
address@hidden://en.wikipedia.org/wiki/Key_derivation_function, Key derivation
+function} is used to derive two pairs of encryption and decryption keys for
+AES-256 and TwoFish, as well as initialization vectors and authentication keys
+(for @uref{http://en.wikipedia.org/wiki/HMAC, HMAC}). The HMAC is computed
+over the encrypted payload. Encrypted messages include an iv_seed and the HMAC
+in the header.
+
+Each encrypted message in the CORE service includes a sequence number and a
+timestamp in the encrypted payload. The CORE service remembers the largest
+observed sequence number and a bit-mask which represents which of the previous
+32 sequence numbers were already used. Messages with sequence numbers lower
+than the largest observed sequence number minus 32 are discarded. Messages
+with a timestamp that is less than @code{REKEY_TOLERANCE} off (5 minutes) are
+also discarded. This of course means that system clocks need to be reasonably
+synchronized for peers to be able to communicate. Additionally, as the
+ephemeral key changes every 12h, a peer would not even be able to decrypt
+messages older than 12h.
+
address@hidden Type maps
address@hidden %**end of header
+
+Once an encrypted connection has been established, peers begin to exchange
+type maps. Type maps are used to allow the CORE service to determine which
+(encrypted) connections should be shown to which applications. A type map is
+an array of 65536 bits representing the different types of messages understood
+by applications using the CORE service. Each CORE service maintains this map,
+simply by setting the respective bit for each message type supported by any of
+the applications using the CORE service. Note that bits for message types
+embedded in higher-level protocols (such as MESH) will not be included in
+these type maps.
+
+Typically, the type map of a peer will be sparse. Thus, the CORE service
+attempts to compress its type map using @code{gzip}-style compression
+("deflate") prior to transmission. However, if the compression fails to
+compact the map, the map may also be transmitted without compression
+(resulting in @code{GNUNET_MESSAGE_TYPE_CORE_COMPRESSED_TYPE_MAP} or
address@hidden messages respectively). Upon
+receiving a type map, the respective CORE service notifies applications about
+the connection to the other peer if they support any message type indicated in
+the type map (or no message type at all). If the CORE service experience a
+connect or disconnect event from an application, it updates its type map
+(setting or unsetting the respective bits) and notifies its neighbours about
+the change. The CORE services of the neighbours then in turn generate connect
+and disconnect events for the peer that sent the type map for their respective
+applications. As CORE messages may be lost, the CORE service confirms
+receiving a type map by sending back a
address@hidden If such a confirmation (with
+the correct hash of the type map) is not received, the sender will retransmit
+the type map (with exponential back-off).
+
address@hidden GNUnet's CADET subsystem
@c %**end of header
address@hidden Top
-
-
-
The CADET subsystem in GNUnet is responsible for secure end-to-end
communications between nodes in the GNUnet overlay network. CADET builds on the
CORE subsystem which provides for the link-layer communication and then adds
@@ -4600,55 +4230,37 @@ routing, forwarding and additional security to the
connections. CADET offers
the same cryptographic services as CORE, but on an end-to-end level. This is
done so peers retransmitting traffic on behalf of other peers cannot access the
payload data.
address@hidden @bullet
-
address@hidden @bullet
@item CADET provides confidentiality with so-called perfect forward secrecy; we
use ECDHE powered by Curve25519 for the key exchange and then use symmetric
encryption, encrypting with both AES-256 and Twofish
-
@item authentication is achieved by signing the ephemeral keys using Ed25519, a
deterministic variant of ECDSA
-
@item integrity protection (using SHA-512 to do encrypt-then-MAC, although only
256 bits are sent to reduce overhead)
-
@item replay protection (using nonces, timestamps, challenge-response, message
counters and ephemeral keys)
-
address@hidden liveness (keep-alive messages, timeout) @end itemize
-
address@hidden liveness (keep-alive messages, timeout)
address@hidden itemize
Additional to the CORE-like security benefits, CADET offers other properties
that make it a more universal service than CORE.
address@hidden @bullet
-
address@hidden @bullet
@item CADET can establish channels to arbitrary peers in GNUnet. If a peer is
not immediately reachable, CADET will find a path through the network and ask
other peers to retransmit the traffic on its behalf.
-
@item CADET offers (optional) reliability mechanisms. In a reliable channel
traffic is guaranteed to arrive complete, unchanged and in-order.
-
@item CADET takes care of flow and congestion control mechanisms, not allowing
the sender to send more traffic than the receiver or the network are able to
process.
@end itemize
address@hidden @bullet
-
-
address@hidden
-
address@hidden itemize
address@hidden libgnunetcadet
address@hidden libgnunetcadet
@c %**end of header
address@hidden Top
-
-
-
The CADET API (defined in gnunet_cadet_service.h) is the messaging API used by
P2P applications built using GNUnet. It provides applications the ability to
send and receive encrypted messages to any peer participating in GNUnet. The
@@ -4721,19 +4333,10 @@ callbacks given to it at the time of
@code{GNUNET_CADET_connect}.
Finally, when an application no longer wants to use CADET, it should call
@code{GNUNET_CADET_disconnect}, but first all channels and pending
transmissions must be closed (otherwise CADET will complain).
address@hidden @bullet
-
address@hidden
-
address@hidden itemize
address@hidden GNUnet's NSE subsystem
address@hidden GNUnet's NSE subsystem
@c %**end of header
address@hidden Top
-
-
-
NSE stands for Network Size Estimation. The NSE subsystem provides other
subsystems and users with a rough estimate of the number of peers currently
participating in the GNUnet overlay. The computed value is not a precise number
@@ -4744,12 +4347,9 @@ while, the real network size will typically (99.7% of
the time) be in the range
of [2/3 estimate, 3/2 estimate]. We will now give an overview of the algorithm
used to calcualte the estimate; all of the details can be found in this
technical report.
address@hidden Motivation
address@hidden %**end of header
-
address@hidden Top
-
address@hidden Motivation
address@hidden %**end of header
Some subsytems, like DHT, need to know the size of the GNUnet network to
optimize some parameters of their own protocol. The decentralized nature of
@@ -4759,12 +4359,9 @@ number of peers in a system, so far there is none to do
so securely. Other
protocols may allow any malicious peer to manipulate the final result or to
take advantage of the system to perform DoS (Denial of Service) attacks against
the network. GNUnet's NSE protocol avoids these drawbacks.
address@hidden Security
address@hidden %**end of header
-
address@hidden Top
-
address@hidden Security
address@hidden %**end of header
The NSE subsystem is designed to be resilient against these attacks. It uses
@uref{http://en.wikipedia.org/wiki/Proof-of-work_system, proofs of work} to
@@ -4774,24 +4371,18 @@ protection comes from the time-based nature of the
protocol: the estimates are
calculated periodically and out-of-time traffic is either ignored or stored for
later retransmission by benign peers. In particular, peers cannot trigger
global network communication at will.
address@hidden Principle
address@hidden %**end of header
-
address@hidden Top
-
address@hidden Principle
address@hidden %**end of header
The algorithm calculates the estimate by finding the globally closest peer ID
to a random, time-based value.
The idea is that the closer the ID is to the random value, the more "densely
packed" the ID space is, and therefore, more peers are in the network.
address@hidden Example
address@hidden %**end of header
-
address@hidden Top
-
address@hidden Example
address@hidden %**end of header
Suppose all peers have IDs between 0 and 100 (our ID space), and the random
value is 42. If the closest peer has the ID 70 we can imagine that the average
@@ -4801,34 +4392,25 @@ can imagine that the space is rather packed with peers,
maybe as much as 50 of
them. Naturally, we could have been rather unlucky, and there is only one peer
and happens to have the ID 44. Thus, the current estimate is calculated as the
average over multiple rounds, and not just a single sample.
address@hidden Algorithm
address@hidden %**end of header
-
address@hidden Top
-
address@hidden Algorithm
address@hidden %**end of header
Given that example, one can imagine that the job of the subsystem is to
efficiently communicate the ID of the closest peer to the target value to all
the other peers, who will calculate the estimate from it.
address@hidden Target value
address@hidden %**end of header
-
address@hidden Top
-
address@hidden Target value
address@hidden %**end of header
The target value itself is generated by hashing the current time, rounded down
to an agreed value. If the rounding amount is 1h (default) and the time is
12:34:56, the time to hash would be 12:00:00. The process is repeated each
rouning amount (in this example would be every hour). Every repetition is
called a round.
address@hidden Timing
address@hidden %**end of header
-
address@hidden Top
-
address@hidden Timing
address@hidden %**end of header
The NSE subsystem has some timing control to avoid everybody broadcasting its
ID all at one. Once each peer has the target random value, it compares its own
@@ -4840,12 +4422,9 @@ is the same as the previous global estimate, its
"broadcast time" will be in
the middle of the round. If its bigger it will be earlier and if its smaler
(the most likely case) it will be later. This ensures that the peers closests
to the target value start broadcasting their ID the first.
address@hidden Controlled Flooding
address@hidden %**end of header
-
address@hidden Top
-
address@hidden Controlled Flooding
address@hidden %**end of header
When a peer receives a value, first it verifies that it is closer than the
closest value it had so far, otherwise it answers the incoming message with a
@@ -4859,13 +4438,9 @@ correct broadcast time comes. This prevents unnecessary
traffic of sub-optimal
values, since a better value can come before the broadcast time, rendering the
previous one obsolete and saving the traffic that would have been used to
broadcast it to the neighbors.
address@hidden Calculating the estimate
address@hidden %**end of header
-
-
address@hidden Top
-
address@hidden Calculating the estimate
address@hidden %**end of header
Once the closest ID has been spread across the network each peer gets the exact
distance betweed this ID and the target value of the round and calculates the
@@ -4878,19 +4453,10 @@ them, giving a result of which usually has one bit of
uncertainty (the real
size could be half of the estimate or twice as much). Note that the actual
network size is calculated in powers of two of the raw input, thus one bit of
uncertainty means a factor of two in the size estimate.
address@hidden @bullet
-
-
address@hidden
address@hidden itemize
address@hidden libgnunetnse
address@hidden libgnunetnse
@c %**end of header
address@hidden Top
-
-
-
The NSE subsystem has the simplest API of all services, with only two calls:
@code{GNUNET_NSE_connect} and @code{GNUNET_NSE_disconnect}.
@@ -4903,12 +4469,9 @@ the first. The default round time is set to 1 hour.
The disconnect call disconnects from the NSE subsystem and the callback is no
longer called with new estimates.
address@hidden Results
address@hidden %**end of header
-
address@hidden Top
-
address@hidden Results
address@hidden %**end of header
The callback provides two values: the average and the
@uref{http://en.wikipedia.org/wiki/Standard_deviation, standard deviation} of
@@ -4916,49 +4479,38 @@ the last 64 rounds. The values provided by the callback
function are
logarithmic, this means that the real estimate numbers can be obtained by
calculating 2 to the power of the given value (2average). From a statistics
point of view this means that:
address@hidden @bullet
-
address@hidden @bullet
@item 68% of the time the real size is included in the interval
[(2average-stddev), 2]
-
@item 95% of the time the real size is included in the interval
[(2average-2*stddev, 2^average+2*stddev]
-
@item 99.7% of the time the real size is included in the interval
[(2average-3*stddev, 2average+3*stddev]
@end itemize
-
The expected standard variation for 64 rounds in a network of stable size is
0.2. Thus, we can say that normally:
address@hidden @bullet
-
address@hidden @bullet
@item 68% of the time the real size is in the range [-13%, +15%]
-
@item 95% of the time the real size is in the range [-24%, +32%]
-
@item 99.7% of the time the real size is in the range [-34%, +52%]
@end itemize
-
As said in the introduction, we can be quite sure that usually the real size is
between one third and three times the estimate. This can of course vary with
network conditions. Thus, applications may want to also consider the provided
standard deviation value, not only the average (in particular, if the standard
veriation is very high, the average maybe meaningless: the network size is
changing rapidly).
address@hidden Examples
address@hidden %**end of header
-
address@hidden Top
-
address@hidden Examples
address@hidden %**end of header
Let's close with a couple examples.
address@hidden @asis
address@hidden @asis
@item Average: 10, std dev: 1 Here the estimate would be 2^10 = 1024 peers.@
The range in which we can be 95% sure is: [2^8, 2^12] = [256, 4096]. We can be
very (>99.7%) sure that the network is not a hundred peers and absolutely sure
@@ -4970,53 +4522,31 @@ sure that the network size is around four million, with
absolutely way of it
being 1 million.
@end table
-
To put this in perspective, if someone remembers the LHC Higgs boson results,
were announced with "5 sigma" and "6 sigma" certainties. In this case a 5 sigma
minimum would be 2 million and a 6 sigma minimum, 1.8 million.
address@hidden
address@hidden
-
address@hidden
-
address@hidden itemize
address@hidden The NSE Client-Service Protocol
address@hidden The NSE Client-Service Protocol
@c %**end of header
address@hidden Top
-
-
-
As with the API, the client-service protocol is very simple, only has 2
different messages, defined in @code{src/nse/nse.h}:
address@hidden @bullet
-
address@hidden @bullet
@item @code{GNUNET_MESSAGE_TYPE_NSE_START}@ This message has no parameters and
is sent from the client to the service upon connection.
-
@item @code{GNUNET_MESSAGE_TYPE_NSE_ESTIMATE}@ This message is sent from the
service to the client for every new estimate and upon connection. Contains a
timestamp for the estimate, the average and the standard deviation for the
-respective round. @end itemize
-
+respective round.
address@hidden itemize
When the @code{GNUNET_NSE_disconnect} API call is executed, the client simply
disconnects from the service, with no message involved.
address@hidden @bullet
-
address@hidden
-
address@hidden itemize
address@hidden The NSE Peer-to-Peer Protocol
address@hidden The NSE Peer-to-Peer Protocol
@c %**end of header
address@hidden Top
-
-
-
The NSE subsystem only has one message in the P2P protocol, the
@code{GNUNET_MESSAGE_TYPE_NSE_P2P_FLOOD} message.
@@ -5034,87 +4564,64 @@ us. The message for the next round is simply stored
until our system clock
advances to the next round. The message for the current round is what we are
flooding the network with right now. At the beginning of each round the peer
does the following:
address@hidden @bullet
-
address@hidden @bullet
@item calculates his own distance to the target value
-
@item creates, signs and stores the message for the current round (unless it
has a better message in the "next round" slot which came early in the previous
round)
-
@item calculates, based on the stored round message (own or received) when to
-stard flooding it to its neighbors @end itemize
-
+stard flooding it to its neighbors
address@hidden itemize
Upon receiving a message the peer checks the validity of the message (round,
proof of work, signature). The next action depends on the contents of the
incoming message:
address@hidden @bullet
-
address@hidden @bullet
@item if the message is worse than the current stored message, the peer sends
the current message back immediately, to stop the other peer from spreading
suboptimal results
-
@item if the message is better than the current stored message, the peer stores
the new message and calculates the new target time to start spreading it to its
neighbors (excluding the one the message came from)
-
@item if the message is for the previous round, it is compared to the message
stored in the "previous round slot", which may then be updated
-
@item if the message is for the next round, it is compared to the message
-stored in the "next round slot", which again may then be updated @end itemize
-
+stored in the "next round slot", which again may then be updated
address@hidden itemize
Finally, when it comes to send the stored message for the current round to the
neighbors there is a random delay added for each neighbor, to avoid traffic
spikes and minimize cross-messages.
address@hidden @bullet
-
-
address@hidden
-
address@hidden itemize @settitle GNUnet's HOSTLIST subsystem @c %**end of header
-
address@hidden Top
-
address@hidden GNUnet's HOSTLIST subsystem
address@hidden %**end of header
Peers in the GNUnet overlay network need address information so that they can
connect with other peers. GNUnet uses so called HELLO messages to store and
exchange peer addresses. GNUnet provides several methods for peers to obtain
this information:
address@hidden @bullet
-
address@hidden @bullet
@item out-of-band exchange of HELLO messages (manually, using for example
gnunet-peerinfo)
-
@item HELLO messages shipped with GNUnet (automatic with distribution)
-
@item UDP neighbor discovery in LAN (IPv4 broadcast, IPv6 multicast)
-
@item topology gossiping (learning from other peers we already connected to),
and
-
@item the HOSTLIST daemon covered in this section, which is particularly
relevant for bootstrapping new peers.
@end itemize
-
New peers have no existing connections (and thus cannot learn from gossip among
peers), may not have other peers in their LAN and might be started with an
outdated set of HELLO messages from the distribution. In this case, getting new
peers to connect to the network requires either manual effort or the use of a
HOSTLIST to obtain HELLOs.
address@hidden HELLOs
address@hidden %**end of header
-
address@hidden Top
-
address@hidden HELLOs
address@hidden %**end of header
The basic information peers require to connect to other peers are contained in
so called HELLO messages you can think of as a business card. Besides the
@@ -5122,13 +4629,9 @@ identity of the peer (based on the cryptographic public
key) a HELLO message
may contain address information that specifies ways to contact a peer. By
obtaining HELLO messages, a peer can learn how to contact other peers.
address@hidden Overview for the HOSTLIST subsystem
address@hidden Overview for the HOSTLIST subsystem
@c %**end of header
address@hidden Top
-
-
-
The HOSTLIST subsystem provides a way to distribute and obtain contact
information to connect to other peers using a simple HTTP GET request. It's
implementation is split in three parts, the main file for the daemon itself
@@ -5141,49 +4644,34 @@ the local peer for download. The client component is
basically a HTTP client
hostlist format is a binary blob containing a sequence of HELLO messages. Note
that any HTTP server can theoretically serve a hostlist, the build-in hostlist
server makes it simply convenient to offer this service.
address@hidden Features
address@hidden %**end of header
-
address@hidden Top
-
address@hidden Features
address@hidden %**end of header
The HOSTLIST daemon can:
address@hidden @bullet
-
address@hidden @bullet
@item provide HELLO messages with validated addresses obtained from PEERINFO to
download for other peers
-
@item download HELLO messages and forward these message to the TRANSPORT
subsystem for validation
-
@item advertises the URL of this peer's hostlist address to other peers via
gossip
-
@item automatically learn about hostlist servers from the gossip of other peers
@end itemize
address@hidden Limitations
address@hidden %**end of header
-
address@hidden Top
-
address@hidden Limitations
address@hidden %**end of header
The HOSTLIST daemon does not:
address@hidden @bullet
-
address@hidden @bullet
@item verify the cryptographic information in the HELLO messages
-
@item verify the address information in the HELLO messages
@end itemize
address@hidden Interacting with the HOSTLIST daemon
address@hidden %**end of header
-
address@hidden Top
-
address@hidden Interacting with the HOSTLIST daemon
address@hidden %**end of header
The HOSTLIST subsystem is currently implemented as a daemon, so there is no
need for the user to interact with it and therefore there is no command line
@@ -5197,7 +4685,6 @@ information about the status of HOSTLIST:
$ gnunet-statistics -s hostlist
@end example
-
In particular, HOSTLIST includes a @strong{persistent} value in statistics that
specifies when the hostlist server might be queried next. As this value is
exponentially increasing during runtime, developers may want to reset or
@@ -5205,12 +4692,9 @@ manually adjust it. Note that HOSTLIST (but not
STATISTICS) needs to be
shutdown if changes to this value are to have any effect on the daemon (as
HOSTLIST does not monitor STATISTICS for changes to the download
frequency).
address@hidden Hostlist security: address validation
address@hidden %**end of header
-
address@hidden Top
-
address@hidden Hostlist security: address validation
address@hidden %**end of header
Since information obtained from other parties cannot be trusted without
validation, we have to distinguish between @emph{validated} and @emph{not
@@ -5225,12 +4709,9 @@ and provides it to other peers. When acting as a client,
it contacts the
HOSTLIST servers specified in the configuration, downloads the (unvalidated)
list of HELLO messages and forwards these information to the TRANSPORT server
to validate the addresses.
address@hidden The HOSTLIST daemon
address@hidden %**end of header
-
address@hidden Top
-
address@hidden The HOSTLIST daemon
address@hidden %**end of header
The hostlist daemon is the main component of the HOSTLIST subsystem. It is
started by the ARM service and (if configured) starts the HOSTLIST client and
@@ -5254,25 +4735,18 @@ events.
To clean up on shutdown, the daemon has a cleaning task, shutting down all
subsystems and disconnecting from CORE.
address@hidden The HOSTLIST server
address@hidden %**end of header
-
-
address@hidden Top
-
address@hidden The HOSTLIST server
address@hidden %**end of header
The server provides a way for other peers to obtain HELLOs. Basically it is a
small web server other peers can connect to and download a list of HELLOs using
standard HTTP; it may also advertise the URL of the hostlist to other peers
connecting on CORE level.
address@hidden The HTTP Server
+
address@hidden The HTTP Server
@c %**end of header
address@hidden Top
-
-
-
During startup, the server starts a web server listening on the port specified
with the HTTPPORT value (default 8080). In addition it connects to the PEERINFO
service to obtain peer information. The HOSTLIST server uses the
@@ -5291,23 +4765,17 @@ A HOSTLIST client connecting to the HOSTLIST server
will receive the hostlist
as a HTTP response and the the server will terminate the connection with the
result code HTTP 200 OK. The connection will be closed immediately if no
hostlist is available.
address@hidden Advertising the URL
address@hidden %**end of header
-
address@hidden Top
-
address@hidden Advertising the URL
address@hidden %**end of header
The server also advertises the URL to download the hostlist to other peers if
hostlist advertisement is enabled. When a new peer connects and has hostlist
learning enabled, the server sends a GNUNET_MESSAGE_TYPE_HOSTLIST_ADVERTISEMENT
message to this peer using the CORE service.
address@hidden The HOSTLIST client
address@hidden %**end of header
-
address@hidden Top
-
address@hidden The HOSTLIST client
address@hidden %**end of header
The client provides the functionality to download the list of HELLOs from a set
of URLs. It performs a standard HTTP request to the URLs configured and learned
@@ -5317,12 +4785,9 @@ validation.
The client supports two modes of operation: download of HELLOs (bootstrapping)
and learning of URLs.
address@hidden Bootstrapping
address@hidden %**end of header
-
address@hidden Top
-
address@hidden Bootstrapping
address@hidden %**end of header
For bootstrapping, it schedules a task to download the hostlist from the set of
known URLs. The downloads are only performed if the number of current
@@ -5341,12 +4806,9 @@ hostlist download is not exceeded
(MAX_BYTES_PER_HOSTLISTS = 500000). When a
full HELLO was downloaded, the HOSTLIST client offers this HELLO message to the
TRANSPORT service for validation. When the download is finished or failed,
statistical information about the quality of this URL is updated.
address@hidden Learning
address@hidden %**end of header
-
address@hidden Top
-
address@hidden Learning
address@hidden %**end of header
The client also manages hostlist advertisements from other peers. The HOSTLIST
daemon forwards GNUNET_MESSAGE_TYPE_HOSTLIST_ADVERTISEMENT messages to the
@@ -5359,12 +4821,9 @@ and the list is full, the URL with the worst quality
ranking (determined
through successful downloads and number of HELLOs e.g.) is discarded. During
shutdown the list of URLs is saved to a file for persistance and loaded on
startup. URLs from the configuration file are never discarded.
address@hidden Usage
address@hidden %**end of header
-
address@hidden Top
-
address@hidden Usage
address@hidden %**end of header
To start HOSTLIST by default, it has to be added to the DEFAULTSERVICES section
for the ARM services. This is done in the default configuration.
@@ -5372,19 +4831,10 @@ for the ARM services. This is done in the default
configuration.
For more information on how to configure the HOSTLIST subsystem see the
installation handbook:@ Configuring the hostlist to bootstrap@ Configuring your
peer to provide a hostlist
address@hidden @bullet
-
address@hidden
-
address@hidden itemize
address@hidden GNUnet's IDENTITY subsystem
address@hidden GNUnet's IDENTITY subsystem
@c %**end of header
address@hidden Top
-
-
-
Identities of "users" in GNUnet are called egos. Egos can be used as pseudonyms
(fake names) or be tied to an organization (for example, GNU) or even the
actual identity of a human. GNUnet users are expected to have many egos. They
@@ -5423,63 +4873,44 @@ The anonymous ego is special in that its private key is
not really private, but
fixed and known to everyone. Thus, anyone can perform actions as anonymous.
This can be useful as with this trick, code does not have to contain a special
case to distinguish between anonymous and pseudonymous egos.
address@hidden @bullet
-
-
address@hidden
address@hidden itemize
address@hidden libgnunetidentity
address@hidden libgnunetidentity
@c %**end of header
address@hidden Top
-
address@hidden Connecting to the service
address@hidden Connecting to the service
@c %**end of header
address@hidden Top
-
-
-
First, typical clients connect to the identity service using
@code{GNUNET_IDENTITY_connect}. This function takes a callback as a parameter.
If the given callback parameter is non-null, it will be invoked to notify the
application about the current state of the identities in the system.
address@hidden
address@hidden
-
address@hidden @bullet
@item First, it will be invoked on all known egos at the time of the
connection. For each ego, a handle to the ego and the user's name for the ego
will be passed to the callback. Furthermore, a @code{void **} context argument
will be provided which gives the client the opportunity to associate some state
with the ego.
-
@item Second, the callback will be invoked with NULL for the ego, the name and
the context. This signals that the (initial) iteration over all egos has
completed.
-
@item Then, the callback will be invoked whenever something changes about an
ego. If an ego is renamed, the callback is invoked with the ego handle of the
ego that was renamed, and the new name. If an ego is deleted, the callback is
invoked with the ego handle and a name of NULL. In the deletion case, the
application should also release resources stored in the context.
-
@item When the application destroys the connection to the identity service
using @code{GNUNET_IDENTITY_disconnect}, the callback is again invoked with the
ego and a name of NULL (equivalent to deletion of the egos). This should again
-be used to clean up the per-ego context. @end itemize
-
+be used to clean up the per-ego context.
address@hidden itemize
The ego handle passed to the callback remains valid until the callback is
invoked with a name of NULL, so it is safe to store a reference to the ego's
handle.
address@hidden Operations on Egos
address@hidden %**end of header
-
address@hidden Top
-
address@hidden Operations on Egos
address@hidden %**end of header
Given an ego handle, the main operations are to get its associated private key
using @code{GNUNET_IDENTITY_ego_get_private_key} or its associated public key
@@ -5497,12 +4928,9 @@ applications that are connected with the identity
service at the time.
respective continuations would be called. It is not guaranteed that the
operation will not be completed anyway, only the continuation will no longer be
called.
address@hidden The anonymous Ego
address@hidden %**end of header
-
address@hidden Top
-
address@hidden The anonymous Ego
address@hidden %**end of header
A special way to obtain an ego handle is to call
@code{GNUNET_IDENTITY_ego_get_anonymous}, which returns an ego for the
@@ -5510,12 +4938,9 @@ A special way to obtain an ego handle is to call
it is suitable for operations that are supposed to be anonymous but require
signatures (for example, to avoid a special path in the code). The anonymous
ego is always valid and accessing it does not require a connection to the
-identity address@hidden Convenience API to lookup a single ego @c %**end of
-header
-
address@hidden Top
-
+identity service.
address@hidden Convenience API to lookup a single ego
As applications commonly simply have to lookup a single ego, there is a
convenience API to do just that. Use @code{GNUNET_IDENTITY_ego_lookup} to
@@ -5524,30 +4949,17 @@ the service function. The resulting ego will be
returned via a callback and
will only be valid during that callback. The operation can be cancelled via
@code{GNUNET_IDENTITY_ego_lookup_cancel} (cancellation is only legal before the
callback is invoked).
address@hidden Associating egos with service functions
address@hidden %**end of header
-
address@hidden Top
-
address@hidden Associating egos with service functions
The @code{GNUNET_IDENTITY_set} function is used to associate a particular ego
with a service function. The name used by the service and the ego are given as
arguments. Afterwards, the service can use its name to lookup the associated
ego using @code{GNUNET_IDENTITY_get}.
address@hidden @bullet
-
-
address@hidden
address@hidden itemize
address@hidden The IDENTITY Client-Service Protocol
address@hidden The IDENTITY Client-Service Protocol
@c %**end of header
address@hidden Top
-
-
-
A client connecting to the identity service first sends a message with type
@code{GNUNET_MESSAGE_TYPE_IDENTITY_START} to the service. After that, the
client will receive information about changes to the egos by receiving messages
@@ -5570,20 +4982,10 @@ private key of the ego. Such bindings can then be
resolved using a GET_DEFAULT
message, which includes the name of the service function. The identity service
will respond to a GET_DEFAULT request with a SET_DEFAULT message containing the
respective information, or with a RESULT_CODE to indicate an error.
address@hidden
address@hidden
-
address@hidden
-
address@hidden itemize
address@hidden GNUnet's NAMESTORE Subsystem
address@hidden GNUnet's NAMESTORE Subsystem
@c %**end of header
address@hidden Top
-
-
-
The NAMESTORE subsystem provides persistent storage for local GNS zone
information. All local GNS zone information are managed by NAMESTORE. It
provides both the functionality to administer local GNS information (e.g.
@@ -5606,19 +5008,10 @@ NAMESTORE provides functionality to look-up and store
records, to iterate over
a specific or all zones and to monitor zones for changes. NAMESTORE
functionality can be accessed using the NAMESTORE api or the NAMESTORE command
line tool.
address@hidden @bullet
-
address@hidden
-
address@hidden itemize
address@hidden libgnunetnamestore
address@hidden libgnunetnamestore
@c %**end of header
address@hidden Top
-
-
-
To interact with NAMESTORE clients first connect to the NAMESTORE service using
the @code{GNUNET_NAMESTORE_connect} passing a configuration handle. As a result
they obtain a NAMESTORE handle, they can use for operations, or NULL is
@@ -5631,12 +5024,9 @@ NAMESTORE internally uses the ECDSA private key to refer
to zones. These
private keys can be obtained from the IDENTITY subsytem. Here @address@hidden
can be used to refer to zones or the default ego assigned to the GNS subsystem
can be used to obtained the master zone's private key.}}
address@hidden Editing Zone Information
address@hidden %**end of header
-
address@hidden Top
-
address@hidden Editing Zone Information
address@hidden %**end of header
NAMESTORE provides functions to lookup records stored under a label in a zone
and to store records under a label in a zone.
@@ -5668,12 +5058,9 @@ and records stored in a zone. Here the client uses the
@code{GNUNET_NAMESTORE_set_nick} function and passes the private key of the
zone, the nickname as string plus a the callback with the result of the
operation.
address@hidden Iterating Zone Information
address@hidden %**end of header
-
address@hidden Top
-
address@hidden Iterating Zone Information
address@hidden %**end of header
A client can iterate over all information in a zone or all zones managed by
NAMESTORE. Here a client uses the @code{GNUNET_NAMESTORE_zone_iteration_start}
@@ -5687,12 +5074,9 @@ NAMESTORE calls the callback for every result and
expects the client to call@
@code{GNUNET_NAMESTORE_zone_iterator_stop} to interrupt the iteration. When
NAMESTORE reached the last item it will call the callback with a NULL value to
indicate.
address@hidden Monitoring Zone Information
address@hidden %**end of header
-
address@hidden Top
-
address@hidden Monitoring Zone Information
address@hidden %**end of header
Clients can also monitor zones to be notified about changes. Here the clients
uses the @code{GNUNET_NAMESTORE_zone_monitor_start} function and passes the
@@ -5706,19 +5090,10 @@ zone, the label and the records and their number.
To stop monitoring, the client call @code{GNUNET_NAMESTORE_zone_monitor_stop}
and passes the handle obtained from the function to start the monitoring.
address@hidden @bullet
-
-
address@hidden
address@hidden itemize
address@hidden GNUnet's PEERINFO subsystem
address@hidden GNUnet's PEERINFO subsystem
@c %**end of header
address@hidden Top
-
-
-
The PEERINFO subsystem is used to store verified (validated) information about
known peers in a persistent way. It obtains these addresses for example from
TRANSPORT service which is in charge of address validation. Validation means
@@ -5733,39 +5108,25 @@ providing a generic persistent per-peer information
store. More and more
subsystems tend to need to store per-peer information in persistent way. To not
duplicate this functionality we plan to provide a PEERSTORE service providing
this functionality
address@hidden Features
address@hidden %**end of header
-
address@hidden Top
address@hidden Features
address@hidden %**end of header
@itemize @bullet
-
-
@item Persistent storage
-
@item Client notification mechanism on update
-
@item Periodic clean up for expired information
-
@item Differentiation between public and friend-only HELLO
@end itemize
address@hidden Limitations @c %**end of header
-
address@hidden Top
address@hidden Limitations @c %**end of header
@itemize @bullet
-
-
@item Does not perform HELLO validation
@end itemize
address@hidden Peer Information
address@hidden %**end of header
-
address@hidden Top
-
address@hidden Peer Information
address@hidden %**end of header
The PEERINFO subsystem stores these information in the form of HELLO messages
you can think of as business cards. These HELLO messages contain the public key
@@ -5790,12 +5151,9 @@ systems can obtain these information from PEERINFO and
use it for their
purposes. Clients are for example the HOSTLIST component providing these
information to other peers in form of a hostlist or the TRANSPORT subsystem
using these information to maintain connections to other peers.
address@hidden Startup
address@hidden %**end of header
-
address@hidden Top
-
address@hidden Startup
address@hidden %**end of header
During startup the PEERINFO services loads persistent HELLOs from disk. First
PEERINFO parses the directory configured in the HOSTS value of the
@@ -5806,12 +5164,9 @@ used to simplify network bootstrapping by providing
valid peer information with
the distribution. The use of these HELLOs can be prevented by setting the
@code{USE_INCLUDED_HELLOS} in the @code{PEERINFO} configuration section to
@code{NO}. Files containing invalid information are removed.
address@hidden Managing Information
address@hidden %**end of header
-
address@hidden Top
-
address@hidden Managing Information
address@hidden %**end of header
The PEERINFO services stores information about known PEERS and a single HELLO
message for every peer. A peer does not need to have a HELLO if no information
@@ -5827,12 +5182,9 @@ periodic task scans all files in the directory and
recreates the HELLO messages
it finds. Expired TRANSPORT addresses are removed from the HELLO and if the
HELLO does not contain any valid addresses, it is discarded and removed from
disk.
address@hidden Obtaining Information
address@hidden %**end of header
-
address@hidden Top
-
address@hidden Obtaining Information
address@hidden %**end of header
When a client requests information from PEERINFO, PEERINFO performs a lookup
for the respective peer or all peers if desired and transmits this information
@@ -5844,19 +5196,10 @@ To notify clients about changes to PEERINFO
information, PEERINFO maintains a
list of clients interested in this notifications. Such a notification occurs if
a HELLO for a peer was updated (due to a merge for example) or a new peer was
added.
address@hidden @bullet
-
address@hidden
-
address@hidden itemize
address@hidden The PEERINFO Client-Service Protocol
address@hidden The PEERINFO Client-Service Protocol
@c %**end of header
address@hidden Top
-
-
-
To connect and disconnect to and from the PEERINFO Service PEERINFO utilizes
the util client/server infrastructure, so no special messages types are used
here.
@@ -5881,39 +5224,24 @@ to transmit with a @code{struct ListAllPeersMessage}
with type
message is @code{struct GNUNET_MessageHeader} with type
@code{GNUNET_MESSAGE_TYPE_PEERINFO_INFO}. If the client receives this message,
he can proceed with the next request if any is pending
address@hidden @bullet
-
-
address@hidden
address@hidden itemize
address@hidden libgnunetpeerinfo
address@hidden libgnunetpeerinfo
@c %**end of header
address@hidden Top
-
-
-
The PEERINFO API consists mainly of three different functionalities:
maintaining a connection to the service, adding new information and retrieving
information form the PEERINFO service.
address@hidden Connecting to the Service
address@hidden %**end of header
-
address@hidden Top
-
address@hidden Connecting to the Service
address@hidden %**end of header
To connect to the PEERINFO service the function @code{GNUNET_PEERINFO_connect}
is used, taking a configuration handle as an argument, and to disconnect from
PEERINFO the function @code{GNUNET_PEERINFO_disconnect}, taking the PEERINFO
handle returned from the connect function has to be called.
address@hidden Adding Information
address@hidden %**end of header
-
address@hidden Top
-
address@hidden Adding Information
address@hidden %**end of header
@code{GNUNET_PEERINFO_add_peer} adds a new peer to the PEERINFO subsystem
storage. This function takes the PEERINFO handle as an argument, the HELLO
@@ -5923,12 +5251,9 @@ operation allowing to cancel the operation with the
respective cancel function
@code{GNUNET_PEERINFO_add_peer_cancel}. To retrieve information from PEERINFO
you can iterate over all information stored with PEERINFO or you can tell
PEERINFO to notify if new peer information are available.
address@hidden Obtaining Information
address@hidden %**end of header
-
address@hidden Top
-
address@hidden Obtaining Information
address@hidden %**end of header
To iterate over information in PEERINFO you use @code{GNUNET_PEERINFO_iterate}.
This function expects the PEERINFO handle, a flag if HELLO messages intended
@@ -5945,42 +5270,25 @@ a flag if friend-only HELLO messages should be
included. The PEERINFO service
will notify you about every change and the callback function will be called to
notify you about changes. The function returns a handle to cancel notifications
with @code{GNUNET_PEERINFO_notify_cancel}.
address@hidden @bullet
-
address@hidden
-
address@hidden itemize
address@hidden GNUnet's PEERSTORE subsystem
address@hidden GNUnet's PEERSTORE subsystem
@c %**end of header
address@hidden Top
-
-
-
GNUnet's PEERSTORE subsystem offers persistent per-peer storage for other
GNUnet subsystems. GNUnet subsystems can use PEERSTORE to persistently store
and retrieve arbitrary data. Each data record stored with PEERSTORE contains
the following fields:
address@hidden @bullet
-
address@hidden @bullet
@item subsystem: Name of the subsystem responsible for the record.
-
@item peerid: Identity of the peer this record is related to.
-
@item key: a key string identifying the record.
-
@item value: binary record value.
-
@item expiry: record expiry date.
@end itemize
address@hidden Functionality
address@hidden %**end of header
-
address@hidden Top
-
address@hidden Functionality
address@hidden %**end of header
Subsystems can store any type of value under a (subsystem, peerid, key)
combination. A "replace" flag set during store operations forces the PEERSTORE
@@ -5990,53 +5298,34 @@ the record is *possibly* deleted by PEERSTORE.
Subsystems can iterate over all values stored under any of the following
combination of fields:
address@hidden @bullet
-
address@hidden @bullet
@item (subsystem)
-
@item (subsystem, peerid)
-
@item (subsystem, key)
-
address@hidden (subsystem, peerid, key) @end itemize
-
address@hidden (subsystem, peerid, key)
address@hidden itemize
Subsystems can also request to be notified about any new values stored under a
(subsystem, peerid, key) combination by sending a "watch" request to
PEERSTORE.
address@hidden Architecture
address@hidden %**end of header
-
address@hidden Top
-
-
-PEERSTORE implements the following components: @itemize @bullet
address@hidden Architecture
address@hidden %**end of header
+PEERSTORE implements the following components:
address@hidden @bullet
@item PEERSTORE service: Handles store, iterate and watch operations.
-
@item PEERSTORE API: API to be used by other subsystems to communicate and
issue commands to the PEERSTORE service.
-
@item PEERSTORE plugins: Handles the persistent storage. At the moment, only an
"sqlite" plugin is implemented.
@end itemize
address@hidden @bullet
-
-
address@hidden
-
address@hidden itemize
address@hidden libgnunetpeerstore
address@hidden libgnunetpeerstore
@c %**end of header
address@hidden Top
-
-
-
libgnunetpeerstore is the library containing the PEERSTORE API. Subsystems
wishing to communicate with the PEERSTORE service use this API to open a
connection to PEERSTORE. This is done by calling
@@ -6072,30 +5361,18 @@ PEERSTORE service. Any pending ITERATE or WATCH
requests will be destroyed. If
the @code{sync_first} flag is set to @code{GNUNET_YES}, the API will delay the
disconnection until all pending STORE requests are sent to the PEERSTORE
service, otherwise, the pending STORE requests will be destroyed as well.
address@hidden @bullet
-
-
address@hidden
address@hidden itemize
address@hidden GNUnet's SET Subsystem
address@hidden GNUnet's SET Subsystem
@c %**end of header
address@hidden Top
-
-
-
The SET service implements efficient set operations between two peers over a
mesh tunnel. Currently, set union and set intersection are the only supported
operations. Elements of a set consist of an @emph{element type} and arbitrary
binary @emph{data}. The size of an element's data is limited to around 62
KB.
address@hidden Local Sets
address@hidden %**end of header
-
address@hidden Top
-
address@hidden Local Sets
address@hidden %**end of header
Sets created by a local client can be modified and reused for multiple
operations. As each set operation requires potentially expensive special
@@ -6104,13 +5381,9 @@ participate in one type of set operation (i.e. union or
intersection). The type
of a set is determined upon its creation. If a the elements of a set are needed
for an operation of a different type, all of the set's element must be copied
to a new set of appropriate type.
address@hidden Set Modifications
address@hidden %**end of header
-
-
address@hidden Top
-
address@hidden Set Modifications
address@hidden %**end of header
Even when set operations are active, one can add to and remove elements from a
set. However, these changes will only be visible to operations that have been
@@ -6118,44 +5391,35 @@ created after the changes have taken place. That is,
every set operation only
sees a snapshot of the set from the time the operation was started. This
mechanism is @emph{not} implemented by copying the whole set, but by attaching
@emph{generation information} to each element and operation.
address@hidden Set Operations
address@hidden %**end of header
-
address@hidden Top
-
address@hidden Set Operations
address@hidden %**end of header
Set operations can be started in two ways: Either by accepting an operation
request from a remote peer, or by requesting a set operation from a remote
peer. Set operations are uniquely identified by the involved @emph{peers}, an
@emph{application id} and the @emph{operation type}.
- The client is notified of incoming set operations by @emph{set listeners}. A
- set listener listens for incoming operations of a specific operation type and
- application id. Once notified of an incoming set request, the client can
- accept the set request (providing a local set for the operation) or reject
- it.
address@hidden Result Elements
address@hidden %**end of header
-
address@hidden Top
-
+The client is notified of incoming set operations by @emph{set listeners}. A
+set listener listens for incoming operations of a specific operation type and
+application id. Once notified of an incoming set request, the client can
+accept the set request (providing a local set for the operation) or reject
+it.
address@hidden Result Elements
address@hidden %**end of header
The SET service has three @emph{result modes} that determine how an operation's
result set is delivered to the client:
address@hidden @bullet
-
address@hidden @bullet
@item @strong{Full Result Set.} All elements of set resulting from the set
operation are returned to the client.
-
@item @strong{Added Elements.} Only elements that result from the operation and
are not already in the local peer's set are returned. Note that for some
operations (like set intersection) this result mode will never return any
elements. This can be useful if only the remove peer is actually interested in
the result of the set operation.
-
@item @strong{Removed Elements.} Only elements that are in the local peer's
initial set but not in the operation's result set are returned. Note that for
some operations (like set union) this result mode will never return any
@@ -6163,24 +5427,12 @@ elements. This can be useful if only the remove peer is
actually interested in
the result of the set operation.
@end itemize
address@hidden @bullet
-
-
address@hidden
-
address@hidden itemize
address@hidden libgnunetset
address@hidden libgnunetset
@c %**end of header
address@hidden Top
-
address@hidden Sets
address@hidden Sets
@c %**end of header
address@hidden Top
-
-
-
New sets are created with @code{GNUNET_SET_create}. Both the local peer's
configuration (as each set has its own client connection) and the operation
type must be specified. The set exists until either the client calls
@@ -6189,11 +5441,10 @@ disrupted. In the latter case, the client is notified
by the return value of
functions dealing with sets. This return value must always be checked.
Elements are added and removed with @code{GNUNET_SET_add_element} and
address@hidden@settitle Listeners @c %**end of header
-
address@hidden Top
-
address@hidden
address@hidden Listeners
address@hidden %**end of header
Listeners are created with @code{GNUNET_SET_listen}. Each time time a remote
peer suggests a set operation with an application id and operation type
@@ -6201,23 +5452,17 @@ matching a listener, the listener's callack is invoked.
The client then must
synchronously call either @code{GNUNET_SET_accept} or @code{GNUNET_SET_reject}.
Note that the operation will not be started until the client calls
@code{GNUNET_SET_commit} (see Section "Supplying a Set").
address@hidden Operations
address@hidden %**end of header
-
address@hidden Top
-
address@hidden Operations
address@hidden %**end of header
Operations to be initiated by the local peer are created with
@code{GNUNET_SET_prepare}. Note that the operation will not be started until
the client calls @code{GNUNET_SET_commit} (see Section "Supplying a
Set").
address@hidden Supplying a Set
address@hidden %**end of header
-
address@hidden Top
-
address@hidden Supplying a Set
address@hidden %**end of header
To create symmetry between the two ways of starting a set operation (accepting
and nitiating it), the operation handles returned by @code{GNUNET_SET_accept}
@@ -6227,12 +5472,9 @@ can not do any work yet.
The client must call @code{GNUNET_SET_commit} to specify a set to use for an
operation. @code{GNUNET_SET_commit} may only be called once per set
operation.
address@hidden The Result Callback
address@hidden %**end of header
-
address@hidden Top
-
address@hidden The Result Callback
address@hidden %**end of header
Clients must specify both a result mode and a result callback with
@code{GNUNET_SET_accept} and @code{GNUNET_SET_prepare}. The result callback
@@ -6241,78 +5483,50 @@ failed or succeeded. The interpretation of the received
element depends on the
result mode. The callback needs to know which result mode it is used in, as the
arguments do not indicate if an element is part of the full result set, or if
it is in the difference between the original set and the final set.
address@hidden
address@hidden
-
-
address@hidden
address@hidden itemize
address@hidden The SET Client-Service Protocol
address@hidden The SET Client-Service Protocol
@c %**end of header
address@hidden Top
-
address@hidden Creating Sets
address@hidden Creating Sets
@c %**end of header
address@hidden Top
-
-
-
For each set of a client, there exists a client connection to the service. Sets
are created by sending the @code{GNUNET_SERVICE_SET_CREATE} message over a new
client connection. Multiple operations for one set are multiplexed over one
client connection, using a request id supplied by the client.
address@hidden Listeners
address@hidden %**end of header
-
address@hidden Top
-
address@hidden Listeners
address@hidden %**end of header
Each listener also requires a seperate client connection. By sending the
@code{GNUNET_SERVICE_SET_LISTEN} message, the client notifies the service of
the application id and operation type it is interested in. A client rejects an
incoming request by sending @code{GNUNET_SERVICE_SET_REJECT} on the listener's
-client connection. In contrast, when accepting an incoming request, a a
address@hidden message must be sent over the@ set that is
-supplied for the set operation.
address@hidden Initiating Operations
address@hidden %**end of header
-
-
address@hidden Top
-
+client connection. In contrast, when accepting an incoming request, a a
address@hidden message must be sent over the@ set that is
+supplied for the set operation.
address@hidden Initiating Operations
address@hidden %**end of header
Operations with remote peers are initiated by sending a
@code{GNUNET_SERVICE_SET_EVALUATE} message to the service. The@ client
connection that this message is sent by determines the set to use.
address@hidden Modifying Sets
address@hidden %**end of header
-
address@hidden Top
-
address@hidden Modifying Sets
address@hidden %**end of header
Sets are modified with the @code{GNUNET_SERVICE_SET_ADD} and
@code{GNUNET_SERVICE_SET_REMOVE} messages.
address@hidden Results and Operation Status
address@hidden %**end of header
-
address@hidden Top
-
address@hidden Results and Operation Status
address@hidden %**end of header
The service notifies the client of result elements and success/failure of a set
operation with the @code{GNUNET_SERVICE_SET_RESULT} message.
address@hidden Iterating Sets
address@hidden %**end of header
-
address@hidden Top
-
address@hidden Iterating Sets
address@hidden %**end of header
All elements of a set can be requested by sending
@code{GNUNET_SERVICE_SET_ITER_REQUEST}. The server responds with
@@ -6320,20 +5534,10 @@ All elements of a set can be requested by sending
with @code{GNUNET_SERVICE_SET_ITER_DONE}. After each received element, the
client@ must send @code{GNUNET_SERVICE_SET_ITER_ACK}. Note that only one set
iteration may be active for a set at any given time.
address@hidden @bullet
-
address@hidden
-
address@hidden itemize
address@hidden The SET-Intersection Peer-to-Peer Protocol
address@hidden The SET-Intersection Peer-to-Peer Protocol
@c %**end of header
-
address@hidden Top
-
-
-
The intersection protocol operates over CADET and starts with a
GNUNET_MESSAGE_TYPE_SET_P2P_OPERATION_REQUEST being sent by the peer initiating
the operation to the peer listening for inbound requests. It includes the
@@ -6353,12 +5557,9 @@ GNUNET_MESSAGE_TYPE_SET_INTERSECTION_P2P_ELEMENT_INFO
response, it beings the
Bloom filter exchange, unless the set size is indicated to be zero, in which
case the intersection is considered finished after just the initial
handshake.
address@hidden The Bloom filter exchange
address@hidden %**end of header
-
address@hidden Top
-
address@hidden The Bloom filter exchange
address@hidden %**end of header
In this phase, each peer transmits a Bloom filter over the remaining keys of
the local set to the other peer using a
@@ -6378,39 +5579,26 @@ the XOR over the keys match what is left of his own
set. If they do, he sends
a@ GNUNET_MESSAGE_TYPE_SET_INTERSECTION_P2P_DONE back to indicate that the
latest set is the final result. Otherwise, the receiver starts another Bloom
fitler exchange, except this time as the sender.
address@hidden Salt
address@hidden %**end of header
-
-
address@hidden Top
-
address@hidden Salt
address@hidden %**end of header
Bloomfilter operations are probablistic: With some non-zero probability the
test may incorrectly say an element is in the set, even though it is not.
- To mitigate this problem, the intersection protocol iterates exchanging Bloom
- filters using a different random 32-bit salt in each iteration (the salt is
- also included in the message). With different salts, set operations may fail
- for different elements. Merging the results from the executions, the
- probability of failure drops to zero.
+To mitigate this problem, the intersection protocol iterates exchanging Bloom
+filters using a different random 32-bit salt in each iteration (the salt is
+also included in the message). With different salts, set operations may fail
+for different elements. Merging the results from the executions, the
+probability of failure drops to zero.
- The iterations terminate once both peers have established that they have sets
- of the same size, and where the XOR over all keys computes the same 512-bit
- value (leaving a failure probability of 2-511).
- @itemize @bullet
+The iterations terminate once both peers have established that they have sets
+of the same size, and where the XOR over all keys computes the same 512-bit
+value (leaving a failure probability of 2-511).
-
address@hidden
-
address@hidden itemize
address@hidden The SET-Union Peer-to-Peer Protocol
address@hidden The SET-Union Peer-to-Peer Protocol
@c %**end of header
address@hidden Top
-
-
-
The SET union protocol is based on Eppstein's efficient set reconciliation
without prior context. You should read this paper first if you want to
understand the protocol.
@@ -6445,19 +5633,10 @@ message instead of another
GNUNET_MESSAGE_TYPE_SET_UNION_P2P_IBF.
All Bloom filter operations use a salt to mingle keys before hasing them into
buckets, such that future iterations have a fresh chance of succeeding if they
failed due to collisions before.
address@hidden @bullet
-
-
address@hidden
address@hidden itemize
address@hidden GNUnet's STATISTICS subsystem
address@hidden GNUnet's STATISTICS subsystem
@c %**end of header
address@hidden Top
-
-
-
In GNUnet, the STATISTICS subsystem offers a central place for all subsystems
to publish unsigned 64-bit integer run-time statistics. Keeping this
information centrally means that there is a unified way for the user to obtain
@@ -6495,19 +5674,10 @@ waits for all the clients that are not monitors to
close their connections
before terminating itself. This is to prevent the loss of data during peer
shutdown --- delaying the STATISTICS service shutdown helps other services to
store important data to STATISTICS during shutdown.
address@hidden @bullet
-
-
address@hidden
address@hidden itemize
address@hidden libgnunetstatistics
address@hidden libgnunetstatistics
@c %**end of header
address@hidden Top
-
-
-
@strong{libgnunetstatistics} is the library containing the API for the
STATISTICS subsystem. Any process requiring to use STATISTICS should use this
API by to open a connection to the STATISTICS service. This is done by calling
@@ -6524,12 +5694,9 @@ under the @code{[STATISTICS]} section in the
configuration. With such a
configuration all calls to @code{GNUNET_STATISTICS_create()} return @code{NULL}
as the STATISTICS subsystem is unavailable and no other functions from the API
can be used.
address@hidden Statistics retrieval
address@hidden %**end of header
-
address@hidden Top
-
address@hidden Statistics retrieval
address@hidden %**end of header
Once a connection to the statistics service is obtained, information about any
other system which uses statistics can be retrieved with the function
@@ -6547,13 +5714,9 @@ Call to @code{GNUNET_STATISTICS_get()} is asynchronous
and can be canceled with
the function @code{GNUNET_STATISTICS_get_cancel()}. This is helpful when
retrieving statistics takes too long and especially when we want to shutdown
and cleanup everything.
address@hidden Setting statistics and updating them
address@hidden %**end of header
-
-
address@hidden Top
-
address@hidden Setting statistics and updating them
address@hidden %**end of header
So far we have seen how to retrieve statistics, here we will learn how we can
set statistics and update them so that other subsystems can retrieve them.
@@ -6575,13 +5738,9 @@ The library will combine multiple set or update
operations into one message if
the client performs requests at a rate that is faster than the available IPC
with the STATISTICS service. Thus, the client does not have to worry about
sending requests too quickly.
address@hidden Watches
address@hidden Watches
@c %**end of header
address@hidden Top
-
-
-
As interesting feature of STATISTICS lies in serving notifications whenever a
statistic of our interest is modified. This is achieved by registering a watch
through the function @code{GNUNET_STATISTICS_watch()}. The parameters of this
@@ -6594,25 +5753,13 @@ the subsystem name and the parameter name cannot be
@code{NULL} in a call to
A registered watch will keep notifying any value changes until
@code{GNUNET_STATISTICS_watch_cancel()} is called with the same parameters that
are used for registering the watch.
address@hidden @bullet
-
-
address@hidden
address@hidden itemize
address@hidden The STATISTICS Client-Service Protocol.
address@hidden The STATISTICS Client-Service Protocol.
@c %**end of header
-
address@hidden Top
-
address@hidden Statistics retrieval
address@hidden Statistics retrieval
@c %**end of header
address@hidden Top
-
-
-
To retrieve statistics, the client transmits a message of type
@code{GNUNET_MESSAGE_TYPE_STATISTICS_GET} containing the given subsystem name
and statistic parameter to the STATISTICS service. The service responds with a
@@ -6620,12 +5767,9 @@ message of type
@code{GNUNET_MESSAGE_TYPE_STATISTICS_VALUE} for each of the
statistics parameters that match the client request for the client. The end of
information retrieved is signaled by the service by sending a message of type
@code{GNUNET_MESSAGE_TYPE_STATISTICS_END}.
address@hidden Setting and updating statistics
address@hidden %**end of header
-
address@hidden Top
-
address@hidden Setting and updating statistics
address@hidden %**end of header
The subsystem name, parameter name, its value and the persistence flag are
communicated to the service through the message
@@ -6643,32 +5787,19 @@ relative update by sending a message of type
@code{GNUNET_MESSAGE_TYPE_STATISTICS_SET} with an update flag
(@code{GNUNET_STATISTICS_SETFLAG_RELATIVE}) signifying that the value in the
message should be treated as an update value.
address@hidden Watching for updates
address@hidden %**end of header
-
address@hidden Top
-
address@hidden Watching for updates
address@hidden %**end of header
The function registers the watch at the service by sending a message of type
@code{GNUNET_MESSAGE_TYPE_STATISTICS_WATCH}. The service then sends
notifications through messages of type
@code{GNUNET_MESSAGE_TYPE_STATISTICS_WATCH_VALUE} whenever the statistic
parameter's value is changed.
address@hidden @bullet
-
address@hidden
-
address@hidden itemize
address@hidden GNUnet's Distributed Hash Table (DHT)
address@hidden GNUnet's Distributed Hash Table (DHT)
@c %**end of header
-
address@hidden Top
-
-
-
GNUnet includes a generic distributed hash table that can be used by developers
building P2P applications in the framework. This section documents high-level
features and how developers are expected to use the DHT. We have a research
@@ -6676,53 +5807,36 @@ paper detailing how the DHT works. Also, Nate's thesis
includes a detailed
description and performance analysis (in chapter 6).
Key features of GNUnet's DHT include:
address@hidden @bullet
-
address@hidden @bullet
@item stores key-value pairs with values up to (approximately) 63k in size
-
@item works with many underlay network topologies (small-world, random graph),
underlay does not need to be a full mesh / clique
-
@item support for extended queries (more than just a simple 'key'), filtering
duplicate replies within the network (bloomfilter) and content validation (for
details, please read the subsection on the block library)
-
@item can (optionally) return paths taken by the PUT and GET operations to the
application
-
@item provides content replication to handle churn
@end itemize
-
- GNUnet's DHT is randomized and unreliable. Unreliable means that there is no
- strict guarantee that a value stored in the DHT is always found --- values are
- only found with high probability. While this is somewhat true in all P2P DHTs,
- GNUnet developers should be particularly wary of this fact (this will help you
- write secure, fault-tolerant code). Thus, when writing any application using
- the DHT, you should always consider the possibility that a value stored in the
- DHT by you or some other peer might simply not be returned, or returned with a
- significant delay. Your application logic must be written to tolerate this
- (naturally, some loss of performance or quality of service is expected in this
+GNUnet's DHT is randomized and unreliable. Unreliable means that there is no
+strict guarantee that a value stored in the DHT is always found --- values are
+only found with high probability. While this is somewhat true in all P2P DHTs,
+GNUnet developers should be particularly wary of this fact (this will help you
+write secure, fault-tolerant code). Thus, when writing any application using
+the DHT, you should always consider the possibility that a value stored in the
+DHT by you or some other peer might simply not be returned, or returned with a
+significant delay. Your application logic must be written to tolerate this
+(naturally, some loss of performance or quality of service is expected in this
case).
address@hidden @bullet
-
-
address@hidden
address@hidden itemize
address@hidden Block library and plugins
address@hidden Block library and plugins
@c %**end of header
address@hidden Top
-
address@hidden What is a Block?
address@hidden What is a Block?
@c %**end of header
address@hidden Top
-
-
-
Blocks are small (< 63k) pieces of data stored under a key (struct
GNUNET_HashCode). Blocks have a type (enum GNUNET_BlockType) which defines
their data format. Blocks are used in GNUnet as units of static data exchanged
@@ -6732,12 +5846,9 @@ stored in blocks) and the DHT (all information in the
DHT and meta-information
for the maintenance of the DHT are both stored using blocks). The block
subsystem provides a few common functions that must be available for any type
of block.
address@hidden The API of libgnunetblock
address@hidden %**end of header
-
address@hidden Top
-
address@hidden The API of libgnunetblock
address@hidden %**end of header
The block library requires for each (family of) block type(s) a block plugin
(implementing gnunet_block_plugin.h) that provides basic functions that are
@@ -6762,12 +5873,9 @@ replies (via the Bloom filter). All plugins MUST support
detecting duplicate
replies (by adding the current response to the Bloom filter and rejecting it if
it is encountered again). If a plugin fails to do this, responses may loop in
the network.
address@hidden Queries
address@hidden %**end of header
-
address@hidden Top
-
address@hidden Queries
address@hidden %**end of header
The query format for any block in GNUnet consists of four main components.
First, the type of the desired block must be specified. Second, the query must
@@ -6788,23 +5896,17 @@ if the query is valid in the first place.
Depending on the results from the plugin, the DHT will then discard the
(invalid) query, forward the query, discard the (invalid) reply, cache the
(valid) reply, and/or forward the (valid and non-duplicate) reply.
address@hidden Sample Code
address@hidden %**end of header
-
address@hidden Top
-
address@hidden Sample Code
address@hidden %**end of header
The source code in @strong{plugin_block_test.c} is a good starting point for
new block plugins --- it does the minimal work by implementing a plugin that
performs no validation at all. The respective @strong{Makefile.am} shows how to
build and install a block plugin.
address@hidden Conclusion
address@hidden %**end of header
-
address@hidden Top
-
address@hidden Conclusion
address@hidden %**end of header
In conclusion, GNUnet subsystems that want to use the DHT need to define a
block format and write a plugin to match queries and replies. For testing, the
@@ -6812,106 +5914,87 @@ block format and write a plugin to match queries and
replies. For testing, the
and any reply as matching any query. This type is also used for the DHT command
line tools. However, it should NOT be used for normal applications due to the
lack of error checking that results from this primitive implementation.
address@hidden @bullet
-
-
address@hidden
-
address@hidden itemize
address@hidden libgnunetdht
address@hidden %**end of header
address@hidden Top
-
-
-
- The DHT API itself is pretty simple and offers the usual GET and PUT functions
- that work as expected. The specified block type refers to the block library
- which allows the DHT to run application-specific logic for data stored in the
- network.
address@hidden GET
address@hidden %**end of header
-
address@hidden Top
-
-
-
- When using GET, the main consideration for developers (other than the block
- library) should be that after issuing a GET, the DHT will continuously cause
- (small amounts of) network traffic until the operation is explicitly canceled.
- So GET does not simply send out a single network request once; instead, the
- DHT will continue to search for data. This is needed to achieve good success
- rates and also handles the case where the respective PUT operation happens
- after the GET operation was started. Developers should not cancel an existing
- GET operation and then explicitly re-start it to trigger a new round of
- network requests; this is simply inefficient, especially as the internal
- automated version can be more efficient, for example by filtering results in
- the network that have already been returned.
-
- If an application that performs a GET request has a set of replies that it
- already knows and would like to filter, it can call@
- @code{GNUNET_DHT_get_filter_known_results} with an array of hashes over the
- respective blocks to tell the DHT that these results are not desired (any
- more). This way, the DHT will filter the respective blocks using the block
- library in the network, which may result in a significant reduction in
- bandwidth consumption.
address@hidden PUT
address@hidden %**end of header
-
address@hidden Top
-
-
-
- In contrast to GET operations, developers @strong{must} manually re-run PUT
- operations periodically (if they intend the content to continue to be
- available). Content stored in the DHT expires or might be lost due to churn.
- Furthermore, GNUnet's DHT typically requires multiple rounds of PUT operations
- before a key-value pair is consistently available to all peers (the DHT
- randomizes paths and thus storage locations, and only after multiple rounds of
- PUTs there will be a sufficient number of replicas in large DHTs). An explicit
- PUT operation using the DHT API will only cause network traffic once, so in
- order to ensure basic availability and resistance to churn (and adversaries),
- PUTs must be repeated. While the exact frequency depends on the application, a
- rule of thumb is that there should be at least a dozen PUT operations within
- the content lifetime. Content in the DHT typically expires after one day, so
- DHT PUT operations should be repeated at least every 1-2 hours.
address@hidden MONITOR
address@hidden %**end of header
-
address@hidden Top
-
-
-
- The DHT API also allows applications to monitor messages crossing the local
- DHT service. The types of messages used by the DHT are GET, PUT and RESULT
- messages. Using the monitoring API, applications can choose to monitor these
- requests, possibly limiting themselves to requests for a particular block
- type.
-
- The monitoring API is not only usefu only for diagnostics, it can also be used
- to trigger application operations based on PUT operations. For example, an
- application may use PUTs to distribute work requests to other peers. The
- workers would then monitor for PUTs that give them work, instead of looking
- for work using GET operations. This can be beneficial, especially if the
- workers have no good way to guess the keys under which work would be stored.
- Naturally, additional protocols might be needed to ensure that the desired
- number of workers will process the distributed workload.
address@hidden DHT Routing Options
address@hidden libgnunetdht
@c %**end of header
address@hidden Top
-
-
+The DHT API itself is pretty simple and offers the usual GET and PUT functions
+that work as expected. The specified block type refers to the block library
+which allows the DHT to run application-specific logic for data stored in the
+network.
- There are two important options for GET and PUT requests: @table @asis
address@hidden GET
address@hidden %**end of header
+
+When using GET, the main consideration for developers (other than the block
+library) should be that after issuing a GET, the DHT will continuously cause
+(small amounts of) network traffic until the operation is explicitly canceled.
+So GET does not simply send out a single network request once; instead, the
+DHT will continue to search for data. This is needed to achieve good success
+rates and also handles the case where the respective PUT operation happens
+after the GET operation was started. Developers should not cancel an existing
+GET operation and then explicitly re-start it to trigger a new round of
+network requests; this is simply inefficient, especially as the internal
+automated version can be more efficient, for example by filtering results in
+the network that have already been returned.
+
+If an application that performs a GET request has a set of replies that it
+already knows and would like to filter, it can call@
address@hidden with an array of hashes over the
+respective blocks to tell the DHT that these results are not desired (any
+more). This way, the DHT will filter the respective blocks using the block
+library in the network, which may result in a significant reduction in
+bandwidth consumption.
+
address@hidden PUT
address@hidden %**end of header
+
+In contrast to GET operations, developers @strong{must} manually re-run PUT
+operations periodically (if they intend the content to continue to be
+available). Content stored in the DHT expires or might be lost due to churn.
+Furthermore, GNUnet's DHT typically requires multiple rounds of PUT operations
+before a key-value pair is consistently available to all peers (the DHT
+randomizes paths and thus storage locations, and only after multiple rounds of
+PUTs there will be a sufficient number of replicas in large DHTs). An explicit
+PUT operation using the DHT API will only cause network traffic once, so in
+order to ensure basic availability and resistance to churn (and adversaries),
+PUTs must be repeated. While the exact frequency depends on the application, a
+rule of thumb is that there should be at least a dozen PUT operations within
+the content lifetime. Content in the DHT typically expires after one day, so
+DHT PUT operations should be repeated at least every 1-2 hours.
+
address@hidden MONITOR
address@hidden %**end of header
+
+The DHT API also allows applications to monitor messages crossing the local
+DHT service. The types of messages used by the DHT are GET, PUT and RESULT
+messages. Using the monitoring API, applications can choose to monitor these
+requests, possibly limiting themselves to requests for a particular block
+type.
+
+The monitoring API is not only usefu only for diagnostics, it can also be used
+to trigger application operations based on PUT operations. For example, an
+application may use PUTs to distribute work requests to other peers. The
+workers would then monitor for PUTs that give them work, instead of looking
+for work using GET operations. This can be beneficial, especially if the
+workers have no good way to guess the keys under which work would be stored.
+Naturally, additional protocols might be needed to ensure that the desired
+number of workers will process the distributed workload.
+
address@hidden DHT Routing Options
address@hidden %**end of header
+
+There are two important options for GET and PUT requests:
address@hidden @asis
@item GNUNET_DHT_RO_DEMULITPLEX_EVERYWHERE This option means that all peers
should process the request, even if their peer ID is not closest to the key.
For a PUT request, this means that all peers that a request traverses may make
a copy of the data. Similarly for a GET request, all peers will check their
local database for a result. Setting this option can thus significantly improve
caching and reduce bandwidth consumption --- at the expense of a larger DHT
-database. If in doubt, we recommend that this option should be used. @item
+database. If in doubt, we recommend that this option should be used.
address@hidden
GNUNET_DHT_RO_RECORD_ROUTE This option instructs the DHT to record the path
that a GET or a PUT request is taking through the overlay network. The
resulting paths are then returned to the application with the respective
@@ -6919,29 +6002,19 @@ result. This allows the receiver of a result to
construct a path to the
originator of the data, which might then be used for routing. Naturally,
setting this option requires additional bandwidth and disk space, so
applications should only set this if the paths are needed by the application
-logic. @item GNUNET_DHT_RO_FIND_PEER This option is an internal option used by
+logic.
address@hidden GNUNET_DHT_RO_FIND_PEER This option is an internal option used by
the DHT's peer discovery mechanism and should not be used by applications.
@item GNUNET_DHT_RO_BART This option is currently not implemented. It may in
-the future offer performance improvements for clique topologies. @end table
-
address@hidden @bullet
-
-
address@hidden
+the future offer performance improvements for clique topologies.
address@hidden table
address@hidden itemize
address@hidden The DHT Client-Service Protocol
address@hidden The DHT Client-Service Protocol
@c %**end of header
address@hidden Top
-
address@hidden PUTting data into the DHT
address@hidden PUTting data into the DHT
@c %**end of header
address@hidden Top
-
-
-
To store (PUT) data into the DHT, the client sends a@ @code{struct
GNUNET_DHT_ClientPutMessage} to the service. This message specifies the block
type, routing options, the desired replication level, the expiration time, key,
@@ -6956,12 +6029,9 @@ client, for example if we detect that the PUT operation
had no effect because
the same key-value pair was already stored in the DHT. However, changing this
would also require additional state and messages in the P2P
interaction.
address@hidden GETting data from the DHT
address@hidden %**end of header
-
address@hidden Top
-
address@hidden GETting data from the DHT
address@hidden %**end of header
To retrieve (GET) data from the DHT, the client sends a@ @code{struct
GNUNET_DHT_ClientGetMessage} to the service. The message specifies routing
@@ -6995,13 +6065,9 @@ is more common as this allows a client to run many
concurrent GET operations
over the same connection with the DHT service --- and to stop them
individually.
address@hidden Monitoring the DHT
address@hidden Monitoring the DHT
@c %**end of header
address@hidden Top
-
-
-
To begin monitoring, the client sends a @code{struct
GNUNET_DHT_MonitorStartStop} message to the DHT service. In this message, flags
can be set to enable (or disable) monitoring of GET, PUT and RESULT messages
@@ -7011,24 +6077,14 @@ service will notify the client about any matching event
using @code{struct
GNUNET_DHT_MonitorGetMessage}s for GET events, @code{struct
GNUNET_DHT_MonitorPutMessage} for PUT events and@ @code{struct
GNUNET_DHT_MonitorGetRespMessage} for RESULTs. Each of these messages contains
-all of the information about the event. @itemize @bullet
+all of the information about the event.
-
address@hidden
-
address@hidden itemize
address@hidden The DHT Peer-to-Peer Protocol
address@hidden The DHT Peer-to-Peer Protocol
@c %**end of header
address@hidden Top
-
address@hidden Routing GETs or PUTs
address@hidden Routing GETs or PUTs
@c %**end of header
address@hidden Top
-
-
-
When routing GETs or PUTs, the DHT service selects a suitable subset of
neighbours for forwarding. The exact number of neighbours can be zero or more
and depends on the hop counter of the query (initially zero) in relation to the
@@ -7037,12 +6093,9 @@ peer's connectivity. Depending on the hop counter and
our network size
estimate, the selection of the peers maybe randomized or by proximity to the
key. Furthermore, requests include a set of peers that a request has already
traversed; those peers are also excluded from the selection.
address@hidden PUTting data into the DHT
address@hidden %**end of header
-
address@hidden Top
-
address@hidden PUTting data into the DHT
address@hidden %**end of header
To PUT data into the DHT, the service sends a @code{struct PeerPutMessage} of
type @code{GNUNET_MESSAGE_TYPE_DHT_P2P_PUT} to the respective neighbour. In
@@ -7057,12 +6110,9 @@ taken so far. The Bloom filter should already contain
the identity of the
previous hop; however, the path should not include the identity of the previous
hop and the receiver should append the identity of the sender to the path, not
its own identity (this is done to reduce bandwidth).
address@hidden GETting data from the DHT
address@hidden %**end of header
-
address@hidden Top
-
address@hidden GETting data from the DHT
address@hidden %**end of header
A peer can search the DHT by sending @code{struct PeerGetMessage}s of type
@code{GNUNET_MESSAGE_TYPE_DHT_P2P_GET} to other peers. In addition to the usual
@@ -7089,19 +6139,11 @@ duplicate results) and when they obtain a matching,
non-filtered response a
Whenver a result is forwarded, the block plugin is used to update the Bloom
filter accordingly, to ensure that the same result is never forwarded more than
once. The DHT service may also cache forwarded results locally if the
-"CACHE_RESULTS" option is set to "YES" in the configuration. @itemize @bullet
+"CACHE_RESULTS" option is set to "YES" in the configuration.
-
address@hidden
-
address@hidden itemize
address@hidden The GNU Name System (GNS)
address@hidden The GNU Name System (GNS)
@c %**end of header
address@hidden Top
-
-
-
The GNU Name System (GNS) is a decentralized database that enables users to
securely resolve names to values. Names can be used to identify other users
(for example, in social networking), or network services (for example, VPN
@@ -7133,64 +6175,57 @@ the correctness of a name-value mapping.
Internally, GNS uses the NAMECACHE to cache information obtained from other
users, the NAMESTORE to store information specific to the local users, and the
DHT to exchange data between users. A plugin API is used to enable applications
-to define new GNS record types. @itemize @bullet
+to define new GNS record types.
-
address@hidden
-
address@hidden itemize
address@hidden libgnunetgns
address@hidden libgnunetgns
@c %**end of header
address@hidden Top
-
-
-
The GNS API itself is extremely simple. Clients first connec to the GNS service
using @code{GNUNET_GNS_connect}. They can then perform lookups using
@code{GNUNET_GNS_lookup} or cancel pending lookups using
@code{GNUNET_GNS_lookup_cancel}. Once finished, clients disconnect using
@code{GNUNET_GNS_disconnect}.
address@hidden Looking up records
address@hidden %**end of header
-
address@hidden Top
-
address@hidden Looking up records
address@hidden %**end of header
address@hidden takes a number of arguments: @table @asis
address@hidden takes a number of arguments:
address@hidden @asis
@item handle This is simply the GNS connection handle from
address@hidden @item name The client needs to specify the name to
-be resolved. This can be any valid DNS or GNS hostname. @item zone The client
address@hidden
address@hidden name The client needs to specify the name to
+be resolved. This can be any valid DNS or GNS hostname.
address@hidden zone The client
needs to specify the public key of the GNS zone against which the resolution
should be done (the ".gnu" zone). Note that a key must be provided, even if the
name ends in ".zkey". This should typically be the public key of the
-master-zone of the user. @item type This is the desired GNS or DNS record type
+master-zone of the user.
address@hidden type This is the desired GNS or DNS record type
to look for. While all records for the given name will be returned, this can be
important if the client wants to resolve record types that themselves delegate
resolution, such as CNAME, PKEY or GNS2DNS. Resolving a record of any of these
types will only work if the respective record type is specified in the request,
as the GNS resolver will otherwise follow the delegation and return the records
-from the respective destination, instead of the delegating record. @item
+from the respective destination, instead of the delegating record.
address@hidden
only_cached This argument should typically be set to @code{GNUNET_NO}. Setting
-it to @code{GNUNET_YES} disables resolution via the overlay network. @item
+it to @code{GNUNET_YES} disables resolution via the overlay network.
address@hidden
shorten_zone_key If GNS encounters new names during resolution, their
respective zones can automatically be learned and added to the "shorten zone".
If this is desired, clients must pass the private key of the shorten zone. If
-NULL is passed, shortening is disabled. @item proc This argument identifies
+NULL is passed, shortening is disabled.
address@hidden proc This argument identifies
the function to call with the result. It is given proc_cls, the number of
records found (possilby zero) and the array of the records as arguments. proc
will only be called once. After proc,> has been called, the lookup must no
-longer be cancelled. @item proc_cls The closure for proc.@
-
+longer be cancelled.
address@hidden proc_cls The closure for proc.
@end table
address@hidden Accessing the records
address@hidden %**end of header
-
address@hidden Top
-
address@hidden Accessing the records
address@hidden %**end of header
The @code{libgnunetgnsrecord} library provides an API to manipulate the GNS
record array that is given to proc. In particular, it offers functions such as
@@ -7200,40 +6235,26 @@ applications.
For DNS records, the @code{libgnunetdnsparser} library provides functions for
parsing (and serializing) common types of DNS records.
address@hidden Creating records
address@hidden %**end of header
-
address@hidden Top
-
address@hidden Creating records
address@hidden %**end of header
Creating GNS records is typically done by building the respective record
information (possibly with the help of @code{libgnunetgnsrecord} and
@code{libgnunetdnsparser}) and then using the @code{libgnunetnamestore} to
publish the information. The GNS API is not involved in this
operation.
address@hidden Future work
address@hidden %**end of header
-
address@hidden Top
-
address@hidden Future work
address@hidden %**end of header
In the future, we want to expand @code{libgnunetgns} to allow applications to
observe shortening operations performed during GNS resolution, for example so
-that users can receive visual feedback when this happens. @itemize @bullet
+that users can receive visual feedback when this happens.
-
address@hidden
-
address@hidden itemize
address@hidden libgnunetgnsrecord
address@hidden libgnunetgnsrecord
@c %**end of header
address@hidden Top
-
-
-
The @code{libgnunetgnsrecord} library is used to manipulate GNS records (in
plaintext or in their encrypted format). Applications mostly interact with
@code{libgnunetgnsrecord} by using the functions to convert GNS record values
@@ -7247,12 +6268,9 @@ We will now discuss the four commonly used functions of
the API.@
@code{libgnunetgnsrecord} does not perform these operations itself, but instead
uses plugins to perform the operation. GNUnet includes plugins to support
common DNS record types as well as standard GNS record types.
address@hidden Value handling
address@hidden %**end of header
-
address@hidden Top
-
address@hidden Value handling
address@hidden %**end of header
@code{GNUNET_GNSRECORD_value_to_string} can be used to convert the (binary)
representation of a GNS record value to a human readable, 0-terminated UTF-8
@@ -7261,11 +6279,10 @@ available plugin.
@code{GNUNET_GNSRECORD_string_to_value} can be used to try to convert a human
readable string to the respective (binary) representation of a GNS record
-value. @settitle Type handling @c %**end of header
-
address@hidden Top
-
+value.
address@hidden Type handling
address@hidden %**end of header
@code{GNUNET_GNSRECORD_typename_to_number} can be used to obtain the numeric
value associated with a given typename. For example, given the typename "A"
@@ -7277,19 +6294,11 @@ time.}
@code{GNUNET_GNSRECORD_number_to_typename} can be used to obtain the typename
associated with a given numeric value. For example, given the type number 1,
-the function will return the typename "A". @itemize @bullet
-
-
address@hidden
+the function will return the typename "A".
address@hidden itemize
address@hidden GNS plugins
address@hidden GNS plugins
@c %**end of header
address@hidden Top
-
-
-
Adding a new GNS record type typically involves writing (or extending) a
GNSRECORD plugin. The plugin needs to implement the
@code{gnunet_gnsrecord_plugin.h} API which provides basic functions that are
@@ -7307,19 +6316,11 @@ supported by the plugin, it should simply return an
error code.
The central functions of the block APIs (plugin and main library) are the same
four functions for converting between values and strings, and typenames and
-numbers documented in the previous section. @itemize @bullet
-
-
address@hidden
+numbers documented in the previous section.
address@hidden itemize
address@hidden The GNS Client-Service Protocol
address@hidden The GNS Client-Service Protocol
@c %**end of header
address@hidden Top
-
-
-
The GNS client-service protocol consists of two simple messages, the
@code{LOOKUP} message and the @code{LOOKUP_RESULT}. Each @code{LOOKUP} message
contains a unique 32-bit identifier, which will be included in the
@@ -7331,20 +6332,11 @@ whether shortening is enabled or networking is
disabled. Finally, the
The response includes the number of records and the records themselves in the
format created by @code{GNUNET_GNSRECORD_records_serialize}. They can thus be
-deserialized using @code{GNUNET_GNSRECORD_records_deserialize}. @itemize
address@hidden
-
-
address@hidden
+deserialized using @code{GNUNET_GNSRECORD_records_deserialize}.
address@hidden itemize
address@hidden Hijacking the DNS-Traffic using gnunet-service-dns
address@hidden Hijacking the DNS-Traffic using gnunet-service-dns
@c %**end of header
address@hidden Top
-
-
-
This section documents how the gnunet-service-dns (and the gnunet-helper-dns)
intercepts DNS queries from the local system.@ This is merely one method for
how we can obtain GNS queries. It is also possible to change @code{resolv.conf}
@@ -7366,12 +6358,9 @@ about previous forward queries. Queries that are not
hijacked by some
application using the DNS service will be sent to the original recipient. The
answer to the query will always be sent back through the virtual interface with
the original nameserver as source address.
address@hidden Network Setup Details
address@hidden %**end of header
-
address@hidden Top
-
address@hidden Network Setup Details
address@hidden %**end of header
The DNS interceptor adds the following rules to the Linux kernel:
@example
@@ -7380,25 +6369,14 @@ ACCEPT iptables -t mangle -I OUTPUT 2 -p udp --dport 53
-j MARK --set-mark 3 ip
rule add fwmark 3 table2 ip route add default via $VIRTUALDNS table2
@end example
+Line 1 makes sure that all packets coming from a port our application opened
+beforehand (@code{$LOCALPORT}) will be routed normally. Line 2 marks every
+other packet to a DNS-Server with mark 3 (chosen arbitrarily). The third line
+adds a routing policy based on this mark 3 via the routing table.
-
- Line 1 makes sure that all packets coming from a port our application opened
- beforehand (@code{$LOCALPORT}) will be routed normally. Line 2 marks every
- other packet to a DNS-Server with mark 3 (chosen arbitrarily). The third line
- adds a routing policy based on this mark 3 via the routing table. @itemize
- @bullet
-
-
address@hidden
-
address@hidden itemize
address@hidden Serving DNS lookups via GNS on W32
address@hidden Serving DNS lookups via GNS on W32
@c %**end of header
address@hidden Top
-
-
-
This section documents how the libw32nsp (and gnunet-gns-helper-service-w32) do
DNS resolutions of DNS queries on the local system. This only applies to GNUnet
running on W32.
@@ -7438,18 +6416,15 @@ one reply, and subsequent calls to
NSPLookupServiceNext() will fail with
WSA_NODATA (first call to NSPLookupServiceNext() might also fail if GNS failed
to find the name, or there was an error connecting to it).
-gnunet-gns-helper-service-w32 does most of the processing: @itemize @bullet
-
+gnunet-gns-helper-service-w32 does most of the processing:
address@hidden @bullet
@item Maintains a connection to GNS.
-
@item Reads GNS config and loads appropriate keys.
-
@item Checks service GUID and decides on the type of record to look up,
refusing to make a lookup outright when unsupported service GUID is passed.
-
address@hidden Launches the lookup @end itemize
-
address@hidden Launches the lookup
address@hidden itemize
When lookup result arrives, gnunet-gns-helper-service-w32 forms a complete
reply (including filling a WSAQUERYSETW structure and, possibly, a binary blob
@@ -7460,19 +6435,11 @@ This works for most normal applications that use
gethostbyname() or
getaddrinfo() to resolve names, but fails to do anything with applications that
use alternative means of resolving names (such as sending queries to a DNS
server directly by themselves). This includes some of well known utilities,
-like "ping" and "nslookup". @itemize @bullet
-
+like "ping" and "nslookup".
address@hidden
-
address@hidden itemize
address@hidden The GNS Namecache
address@hidden The GNS Namecache
@c %**end of header
address@hidden Top
-
-
-
The NAMECACHE subsystem is responsible for caching (encrypted) resolution
results of the GNU Name System (GNS). GNS makes zone information available to
other users via the DHT. However, as accessing the DHT for every lookup is
@@ -7495,19 +6462,10 @@ configuration option to limit the size of the
NAMECACHE. It is assumed to be
always small enough (a few MB) to fit on the drive.
The NAMECACHE supports the use of different database backends via a plugin API.
address@hidden @bullet
-
address@hidden
-
address@hidden itemize
address@hidden libgnunetnamecache
address@hidden libgnunetnamecache
@c %**end of header
address@hidden Top
-
-
-
The NAMECACHE API consists of five simple functions. First, there is
@code{GNUNET_NAMECACHE_connect} to connect to the NAMECACHE service. This
returns the handle required for all other operations on the NAMECACHE. Using
@@ -7522,32 +6480,19 @@ invoked. Finally, @code{GNUNET_NAMECACHE_disconnect}
closes the connection to
the NAMECACHE.
The maximum size of a block that can be stored in the NAMECACHE is
address@hidden, which is defined to be 63 kB. @itemize
address@hidden
address@hidden, which is defined to be 63 kB.
-
address@hidden
-
address@hidden itemize
address@hidden The NAMECACHE Client-Service Protocol
address@hidden The NAMECACHE Client-Service Protocol
@c %**end of header
-
address@hidden Top
-
-
-
All messages in the NAMECACHE IPC protocol start with the @code{struct
GNUNET_NAMECACHE_Header} which adds a request ID (32-bit integer) to the
standard message header. The request ID is used to match requests with the
respective responses from the NAMECACHE, as they are allowed to happen
out-of-order.
address@hidden Lookup
address@hidden %**end of header
-
address@hidden Top
-
address@hidden Lookup
address@hidden %**end of header
The @code{struct LookupBlockMessage} is used to lookup a block stored in the
cache. It contains the query hash. The NAMECACHE always responds with a
@@ -7555,50 +6500,33 @@ cache. It contains the query hash. The NAMECACHE always
responds with a
sets the expiration time in the response to zero. Otherwise, the response is
expected to contain the expiration time, the ECDSA signature, the derived key
and the (variable-size) encrypted data of the block.
address@hidden Store
address@hidden %**end of header
-
-
address@hidden Top
-
address@hidden Store
address@hidden %**end of header
The @code{struct BlockCacheMessage} is used to cache a block in the NAMECACHE.
It has the same structure as the @code{struct LookupBlockResponseMessage}. The
service responds with a @code{struct BlockCacheResponseMessage} which contains
the result of the operation (success or failure). In the future, we might want
-to make it possible to provide an error message as well. @itemize @bullet
-
-
address@hidden
-
address@hidden itemize @settitle The NAMECACHE Plugin API @c %**end of header
-
address@hidden Top
-
+to make it possible to provide an error message as well.
address@hidden The NAMECACHE Plugin API
address@hidden %**end of header
The NAMECACHE plugin API consists of two functions, @code{cache_block} to store
a block in the database, and @code{lookup_block} to lookup a block in the
database.
address@hidden Lookup
address@hidden %**end of header
-
address@hidden Top
-
address@hidden Lookup
address@hidden %**end of header
The @code{lookup_block} function is expected to return at most one block to the
iterator, and return @code{GNUNET_NO} if there were no non-expired results. If
there are multiple non-expired results in the cache, the lookup is supposed to
return the result with the largest expiration time.
address@hidden Store
address@hidden %**end of header
-
-
address@hidden Top
-
address@hidden Store
address@hidden %**end of header
The @code{cache_block} function is expected to try to store the block in the
database, and return @code{GNUNET_SYSERR} if this was not possible for any
@@ -7609,31 +6537,19 @@ another block stored under the same key. In this case,
the plugin must ensure
that the block with the larger expiration time is preserved. Obviously, this
can done either by simply adding new blocks and selecting for the most recent
expiration time during lookup, or by checking which block is more recent during
-the store operation. @itemize @bullet
-
+the store operation.
address@hidden
-
address@hidden itemize
address@hidden The REVOCATION Subsystem
address@hidden The REVOCATION Subsystem
@c %**end of header
address@hidden Top
-
-
-
The REVOCATION subsystem is responsible for key revocation of Egos. If a user
learns that his private key has been compromised or has lost it, he can use the
REVOCATION system to inform all of the other users that this private key is no
longer valid. The subsystem thus includes ways to query for the validity of
keys and to propagate revocation messages.
address@hidden Dissemination
address@hidden %**end of header
-
-
address@hidden Top
-
address@hidden Dissemination
address@hidden %**end of header
When a revocation is performed, the revocation is first of all disseminated by
flooding the overlay network. The goal is to reach every peer, so that when a
@@ -7651,13 +6567,9 @@ performs efficient set reconciliation over the sets of
known revocation
messages whenever two peers (that both support REVOCATION dissemination)
connect. The SET service is used to perform this operation
efficiently.
address@hidden Revocation Message: Design Requirements
address@hidden %**end of header
-
-
address@hidden Top
-
address@hidden Revocation Message: Design Requirements
address@hidden %**end of header
However, flooding is also quite costly, creating O(|E|) messages on a network
with |E| edges. Thus, revocation messages are required to contain a
@@ -7673,39 +6585,24 @@ use instantly after their key has expired.
Revocation messages must also be signed by the private key that is being
revoked. Thus, they can only be created while the private key is in the
possession of the respective user. This is another reason to create a
-revocation message ahead of time and store it in a secure location. @itemize
address@hidden
-
+revocation message ahead of time and store it in a secure location.
address@hidden
-
address@hidden itemize
address@hidden libgnunetrevocation
address@hidden libgnunetrevocation
@c %**end of header
address@hidden Top
-
-
-
The REVOCATION API consists of two parts, to query and to issue
revocations.
address@hidden Querying for revoked keys
address@hidden %**end of header
-
address@hidden Top
-
address@hidden Querying for revoked keys
address@hidden %**end of header
@code{GNUNET_REVOCATION_query} is used to check if a given ECDSA public key has
been revoked. The given callback will be invoked with the result of the check.
The query can be cancelled using @code{GNUNET_REVOCATION_query_cancel} on the
return value.
address@hidden Preparing revocations
address@hidden %**end of header
-
address@hidden Top
-
address@hidden Preparing revocations
address@hidden %**end of header
It is often desirable to create a revocation record ahead-of-time and store it
in an off-line location to be used later in an emergency. This is particularly
@@ -7732,30 +6629,17 @@ is required in a revocation message. It takes the
private key that (possibly in
the future) is to be revoked and returns the signature. The signature can again
be saved to disk for later use, which will then allow performing a revocation
even without access to the private key.
address@hidden Issuing revocations
address@hidden %**end
-of header
-
address@hidden Top
-
address@hidden Issuing revocations
Given a ECDSA public key, the signature from @code{GNUNET_REVOCATION_sign} and
the proof-of-work, @code{GNUNET_REVOCATION_revoke} can be used to perform the
actual revocation. The given callback is called upon completion of the
operation. @code{GNUNET_REVOCATION_revoke_cancel} can be used to stop the
library from calling the continuation; however, in that case it is undefined
-whether or not the revocation operation will be executed. @itemize @bullet
-
-
address@hidden
-
address@hidden itemize @settitle The REVOCATION Client-Service Protocol @c
%**end of
-header
-
address@hidden Top
-
+whether or not the revocation operation will be executed.
address@hidden The REVOCATION Client-Service Protocol
The REVOCATION protocol consists of four simple messages.
@@ -7770,20 +6654,11 @@ ECDSA public key to be revoked, a signature by the
corresponding private key
and the proof-of-work, The service responds with a
@code{RevocationResponseMessage} which can be used to indicate that the
@code{RevokeMessage} was invalid (i.e. proof of work incorrect), or otherwise
-indicates that the revocation has been processed successfully. @itemize
address@hidden
-
+indicates that the revocation has been processed successfully.
address@hidden
-
address@hidden itemize
address@hidden The REVOCATION Peer-to-Peer Protocol
address@hidden The REVOCATION Peer-to-Peer Protocol
@c %**end of header
address@hidden Top
-
-
-
Revocation uses two disjoint ways to spread revocation information among peers.
First of all, P2P gossip exchanged via CORE-level neighbours is used to quickly
spread revocations to all connected peers. Second, whenever two peers (that
@@ -7807,19 +6682,10 @@ The current implementation accepts revocation set union
operations from all
peers at any time; however, well-behaved peers should only initiate this
operation once after establishing a connection to a peer with a larger hashed
peer identity.
address@hidden @bullet
-
address@hidden
-
address@hidden itemize
address@hidden GNUnet's File-sharing (FS) Subsystem
address@hidden GNUnet's File-sharing (FS) Subsystem
@c %**end of header
address@hidden Top
-
-
-
This chapter describes the details of how the file-sharing service works. As
with all services, it is split into an API (libgnunetfs), the service process
(gnunet-service-fs) and user interface(s). The file-sharing service uses the
@@ -7842,86 +6708,59 @@ the block FS plugin and the FS service. ECRS on-demand
encoding is implemented
in the FS service.
NOTE: The documentation in this chapter is quite incomplete.
address@hidden @bullet
-
-
address@hidden
address@hidden itemize
address@hidden Encoding for Censorship-Resistant Sharing (ECRS)
address@hidden Encoding for Censorship-Resistant Sharing (ECRS)
@c %**end of header
address@hidden Top
-
-
+When GNUnet shares files, it uses a content encoding that is called ECRS, the
+Encoding for Censorship-Resistant Sharing. Most of ECRS is described in the
+(so far unpublished) research paper attached to this page. ECRS obsoletes the
+previous ESED and ESED II encodings which were used in GNUnet before version
+0.7.0.@ @ The rest of this page assumes that the reader is familiar with the
+attached paper. What follows is a description of some minor extensions that
+GNUnet makes over what is described in the paper. The reason why these
+extensions are not in the paper is that we felt that they were obvious or
+trivial extensions to the original scheme and thus did not warrant space in
+the research report.
- When GNUnet shares files, it uses a content encoding that is called ECRS, the
- Encoding for Censorship-Resistant Sharing. Most of ECRS is described in the
- (so far unpublished) research paper attached to this page. ECRS obsoletes the
- previous ESED and ESED II encodings which were used in GNUnet before version
- 0.7.0.@ @ The rest of this page assumes that the reader is familiar with the
- attached paper. What follows is a description of some minor extensions that
- GNUnet makes over what is described in the paper. The reason why these
- extensions are not in the paper is that we felt that they were obvious or
- trivial extensions to the original scheme and thus did not warrant space in
- the research report.
address@hidden Namespace Advertisements
address@hidden Namespace Advertisements
@c %**end of header
address@hidden Top
-
+An @code{SBlock} with identifier â²all zerosâ² is a signed
+advertisement for a namespace. This special @code{SBlock} contains metadata
+describing the content of the namespace. Instead of the name of the identifier
+for a potential update, it contains the identifier for the root of the
+namespace. The URI should always be empty. The @code{SBlock} is signed with
+the content provderâ²s RSA private key (just like any other SBlock). Peers
+can search for @code{SBlock}s in order to find out more about a namespace.
-
- An @code{SBlock} with identifier â²all zerosâ² is a signed
- advertisement for a namespace. This special @code{SBlock} contains metadata
- describing the content of the namespace. Instead of the name of the identifier
- for a potential update, it contains the identifier for the root of the
- namespace. The URI should always be empty. The @code{SBlock} is signed with
- the content provderâ²s RSA private key (just like any other SBlock). Peers
- can search for @code{SBlock}s in order to find out more about a namespace.
address@hidden KSBlocks
address@hidden KSBlocks
@c %**end of header
address@hidden Top
-
-
-
- GNUnet implements @code{KSBlocks} which are @code{KBlocks} that, instead of
- encrypting a CHK and metadata, encrypt an @code{SBlock} instead. In other
- words, @code{KSBlocks} enable GNUnet to find @code{SBlocks} using the global
- keyword search. Usually the encrypted @code{SBlock} is a namespace
- advertisement. The rationale behind @code{KSBlock}s and @code{SBlock}s is to
- enable peers to discover namespaces via keyword searches, and, to associate
- useful information with namespaces. When GNUnet finds @code{KSBlocks} during a
- normal keyword search, it adds the information to an internal list of
- discovered namespaces. Users looking for interesting namespaces can then
- inspect this list, reducing the need for out-of-band discovery of namespaces.
- Naturally, namespaces (or more specifically, namespace advertisements) can
- also be referenced from directories, but @code{KSBlock}s should make it easier
- to advertise namespaces for the owner of the pseudonym since they eliminate
- the need to first create a directory.
-
- Collections are also advertised using @code{KSBlock}s. @itemize @bullet
+GNUnet implements @code{KSBlocks} which are @code{KBlocks} that, instead of
+encrypting a CHK and metadata, encrypt an @code{SBlock} instead. In other
+words, @code{KSBlocks} enable GNUnet to find @code{SBlocks} using the global
+keyword search. Usually the encrypted @code{SBlock} is a namespace
+advertisement. The rationale behind @code{KSBlock}s and @code{SBlock}s is to
+enable peers to discover namespaces via keyword searches, and, to associate
+useful information with namespaces. When GNUnet finds @code{KSBlocks} during a
+normal keyword search, it adds the information to an internal list of
+discovered namespaces. Users looking for interesting namespaces can then
+inspect this list, reducing the need for out-of-band discovery of namespaces.
+Naturally, namespaces (or more specifically, namespace advertisements) can
+also be referenced from directories, but @code{KSBlock}s should make it easier
+to advertise namespaces for the owner of the pseudonym since they eliminate
+the need to first create a directory.
-
address@hidden
-
address@hidden itemize
+Collections are also advertised using @code{KSBlock}s.
@table @asis
-
@item Attachment Size
-
@item ecrs.pdf 270.68 KB
-
@end table
address@hidden File-sharing persistence directory structure
address@hidden %**end of header
-
-
address@hidden Top
-
address@hidden File-sharing persistence directory structure
address@hidden %**end of header
This section documents how the file-sharing library implements persistence of
file-sharing operations and specifically the resulting directory structure.
@@ -7964,17 +6803,13 @@ by the name of the client as given to
@code{GNUNET_FS_start} (for example,
"gnunet-gtk") followed by the actual name of the master or child directory.
The names for the master directories follow the names of the operations:
address@hidden @bullet
-
address@hidden @bullet
@item "search"
-
@item "download"
-
@item "publish"
-
address@hidden "unindex" @end itemize
-
address@hidden "unindex"
address@hidden itemize
Each of the master directories contains names (chosen at random) for each
active top-level (master) operation. Note that a download that is associated
@@ -7992,19 +6827,11 @@ publishing operations, the "publish-file" directory
contains information about
the individual files and directories that are part of the publication. However,
this directory structure is flat and does not mirror the structure of the
publishing operation. Note that unindex operations cannot have associated child
-operations. @itemize @bullet
-
-
address@hidden
+operations.
address@hidden itemize
address@hidden GNUnet's REGEX Subsystem
address@hidden GNUnet's REGEX Subsystem
@c %**end of header
address@hidden Top
-
-
-
Using the REGEX subsystem, you can discover peers that offer a particular
service using regular expressions. The peers that offer a service specify it
using a regular expressions. Peers that want to patronize a service search
@@ -8013,16 +6840,10 @@ matching offerers to the patrons.
For the technical details, we have "Max's defense talk and Max's Master's
thesis. An additional publication is under preparation and available to team
-members (in Git). @itemize @bullet
-
-
address@hidden
-
address@hidden itemize @settitle How to run the regex profiler @c %**end of
header
-
address@hidden Top
-
+members (in Git).
address@hidden How to run the regex profiler
address@hidden %**end of header
The gnunet-regex-profiler can be used to profile the usage of mesh/regex for a
given set of regular expressions and strings. Mesh/regex allows you to announce
@@ -8043,8 +6864,11 @@ highlighted.
Announcing of the regular expressions is done by the
gnunet-daemon-regexprofiler, therefore you have to make sure it is started, by
-adding it to the AUTOSTART set of ARM:@ @code{@ [regexprofiler]@ AUTOSTART =
-YES@ }
+adding it to the AUTOSTART set of ARM:@
address@hidden
+[regexprofiler]@
+AUTOSTART = YES@
+}
Furthermore you have to specify the location of the binary:@ @code{@
[regexprofiler]@
@@ -8056,26 +6880,29 @@ Furthermore you have to specify the location of the
binary:@ @code{@
When running the profiler with a large scale deployment, you probably want to
reduce the workload of each peer. Use the following options to do this.@
address@hidden@ [dht]@
- # Force network size estimation@
- FORCE_NSE = 1}
-
-[dhtcache]@ DATABASE = heap@
- # Disable RC-file for Bloom filter? (for benchmarking with limited IO
- # availability)@
- DISABLE_BF_RC = YES@
- # Disable Bloom filter entirely@
- DISABLE_BF = YES
address@hidden
+[dht]@
+# Force network size estimation@
+FORCE_NSE = 1
+
+[dhtcache]
+DATABASE = heap@
+# Disable RC-file for Bloom filter? (for benchmarking with limited IO
+# availability)@
+DISABLE_BF_RC = YES@
+# Disable Bloom filter entirely@
+DISABLE_BF = YES
[nse]@
- # Minimize proof-of-work CPU consumption by NSE@
- WORKBITS = 1@
+# Minimize proof-of-work CPU consumption by NSE@
+WORKBITS = 1}
@strong{Options}
To finally run the profiler some options and the input data need to be
-specified on the command line.@ @code{@ gnunet-regex-profiler -c config-file -d
+specified on the command line.
address@hidden@ gnunet-regex-profiler -c config-file -d
log-file -n num-links -p@ path-compression-length -s search-delay -t
matching-timeout -a num-search-strings hosts-file policy-dir
search-strings-file@ }
diff --git a/gnunet.texi b/gnunet.texi
new file mode 100644
index 0000000..6db79cb
--- /dev/null
+++ b/gnunet.texi
@@ -0,0 +1,76 @@
+\input texinfo
address@hidden -*-texinfo-*-
+
address@hidden %**start of header
address@hidden gnunet.info
address@hidden UTF-8
address@hidden GNUnet Reference Manual
address@hidden %**end of header
+
address@hidden version.texi
+
address@hidden
+Copyright @copyright{} 2017 ng0
+
+Permission is granted to copy, distribute and/or modify this document
+under the terms of the GNU Free Documentation License, Version 1.3 or
+any later version published by the Free Software Foundation; with no
+Invariant Sections, no Front-Cover Texts, and no Back-Cover Texts. A
+copy of the license is included in the section entitled ``GNU Free
+Documentation License''.
+
+A copy of the license is also available from the Free Software
+Foundation Web site at @url{http://www.gnu.org/licenses/fdl.html}.
+
+Alternately, this document is also available under the General
+Public License, version 3 or later, as published by the Free Software
+Foundation. A copy of the license is included in the section entitled
+``GNU General Public License''.
+
+A copy of the license is also available from the Free Software
+Foundation Web site at @url{http://www.gnu.org/licenses/gpl.html}.
address@hidden copying
+
address@hidden
address@hidden GNUnet Reference Manual
address@hidden Using and developing GNUnet
address@hidden The GNUnet Developers
+
address@hidden
address@hidden 0pt plus 1filll
+Edition @value{EDITION} @*
address@hidden @*
+
address@hidden
address@hidden titlepage
+
address@hidden
+
address@hidden
*********************************************************************
address@hidden GNU Free Documentation License
address@hidden GNU Free Documentation License
address@hidden license, GNU Free Documentation License
address@hidden fdl-1.3.texi
+
address@hidden
*********************************************************************
address@hidden GNU General Public License
address@hidden GNU General Public License
address@hidden license, GNU General Public License
address@hidden gpl-3.0.texi
+
address@hidden
*********************************************************************
address@hidden Concept Index
address@hidden Concept Index
address@hidden cp
+
address@hidden Programming Index
address@hidden Programming Index
address@hidden tp fn
address@hidden vr fn
address@hidden fn
+
address@hidden
+
address@hidden Local Variables:
address@hidden ispell-local-dictionary: "american";
address@hidden End:
diff --git a/version.texi b/version.texi
index e6c3e7c..d0a43f1 100644
--- a/version.texi
+++ b/version.texi
@@ -1,4 +1,4 @@
address@hidden UPDATED 22 February 2017
address@hidden UPDATED-MONTH February 2017
address@hidden UPDATED 18 March 2017
address@hidden UPDATED-MONTH March 2017
@set EDITION 0.10.2
@set VERSION 0.10.2
--
To stop receiving notification emails like this one, please contact
address@hidden
| [Prev in Thread] |
Current Thread |
[Next in Thread] |
- [GNUnet-SVN] [gnunet-texinfo] branch master updated: Makefile: PHONY for clean. TODO: document some tasks developer.texi: multitable fixes, batch of settitle -> node changes gnunet.texi: New file in progress version.texi: Should be created by 'make' later on. This is a static copy.,
gnunet <=