[Top][All Lists]
[Date Prev][Date Next][Thread Prev][Thread Next][Date Index][Thread Index]
Re: [GNUnet-developers] more efficient sharing of similar data
From: |
Christian Grothoff |
Subject: |
Re: [GNUnet-developers] more efficient sharing of similar data |
Date: |
Sat, 17 Aug 2002 15:33:17 -0500 |
User-agent: |
KMail/1.4.1 |
-----BEGIN PGP SIGNED MESSAGE-----
Hash: SHA1
On Saturday 17 August 2002 06:25 am, Martin Uecker wrote:
> In the preprocessing case each place where the rolling checksum
> hits the magic value would be moved to a 1k boundary. This scheme
> would work without modifying one line of code in gnunet.
I kind of doubt that - if we take on-demand encoding into consideration (where
GNUnet receives a request and then looks at 1k of the (unmodified) file in
plaintext on the drive and sends back the reply).
> Integrating it into the client has the advantage that the user
> can serve unmodified files (this is an argument for putting ECC
> into the client to). But the index where the hashes are stored
> must be extended with information where each block starts.
> (10 bits of additional information for each 1k block)
You can have 10 bits, and only 10 bits: :-)
/**
* Type of the content index file entries.
**/
typedef struct {
/** the double-hash of this entry */
HashCode160 doubleHash __attribute__ ((packed));
/** the current rating of this content (network byte order)*/
unsigned int importance __attribute__ ((packed));
/** the filename, 0 for entry not used, -1 for data/content shared, > 0 for
on-demand encoding (network byte order)*/
unsigned int fileNameIndex __attribute__ ((packed));
/** the offset in the file, offset for on-demand-encoding (network byte
order) */
unsigned int fileOffset __attribute__ ((packed));
} ContentIndex __attribute__ ((packed));
The last 10 bits in 'fileOffset' are currently always 0 (1k alignment!).
> > | Also I'm not sure how tightly you can fit all blocks into the 1k
> > | scheme (which would of course be best since you gain anonymity if all
> > | blocks are truely uniform in size).
>
> Yes, this is critical for efficiency too.
>
> I have to think about the idea some more but I will provide
> patches to play with next month.
Well, I hope you'll give us a pile of documentation / descriptions with them,
because this sounds like it's going to be *very* complicated (but promising,
nevertheless, could solve one of the problems that we had with our
tree-encoding which was that inserting a single character at the beginning
would make the entire thing look totally different). So I'm looking forward
to this.
Christian
-----BEGIN PGP SIGNATURE-----
Version: GnuPG v1.0.6 (GNU/Linux)
Comment: For info see http://www.gnupg.org
iD8DBQE9XrMN9tNtMeXQLkIRAokiAJ4gSH1Wl3KY/ja8n95eG4rs02kOOACcCAyd
6l04XQsvXrsNuNvt5AyH7l4=
=OWIJ
-----END PGP SIGNATURE-----