[Top][All Lists]
[Date Prev][Date Next][Thread Prev][Thread Next][Date Index][Thread Index]
[GNUnet-SVN] r12734 - in gnunet: . contrib doc src/datastore src/vpn
|
From: |
gnunet |
|
Subject: |
[GNUnet-SVN] r12734 - in gnunet: . contrib doc src/datastore src/vpn |
|
Date: |
Wed, 25 Aug 2010 21:10:18 +0200 |
Author: grothoff
Date: 2010-08-25 21:10:18 +0200 (Wed, 25 Aug 2010)
New Revision: 12734
Added:
gnunet/doc/README.mysql
gnunet/doc/README.postgres
gnunet/src/datastore/perf_plugin_datastore_data_postgres.conf
gnunet/src/datastore/plugin_datastore_postgres.c
gnunet/src/datastore/test_datastore_api_data_postgres.conf
Modified:
gnunet/configure.ac
gnunet/contrib/defaults.conf
gnunet/src/datastore/
gnunet/src/datastore/Makefile.am
gnunet/src/vpn/
Log:
postgres hackery
Modified: gnunet/configure.ac
===================================================================
--- gnunet/configure.ac 2010-08-25 13:52:09 UTC (rev 12733)
+++ gnunet/configure.ac 2010-08-25 19:10:18 UTC (rev 12734)
@@ -330,8 +330,8 @@
CPPFLAGS="-I$with_postgres/include $CPPFLAGS"
AC_CHECK_HEADERS(postgresql/libpq-fe.h,
EXT_LIB_PATH="-L$with_postgres/lib $EXT_LIB_PATH"
- SQLITE_LDFLAGS="-L$with_postgres/lib"
- SQLITE_CPPFLAGS="-I$with_postgres/include"
+ POSTGRES_LDFLAGS="-L$with_postgres/lib"
+ POSTGRES_CPPFLAGS="-I$with_postgres/include"
postgres=true)
LDFLAGS=$SAVE_LDFLAGS
CPPFLAGS=$SAVE_CPPFLAGS
Modified: gnunet/contrib/defaults.conf
===================================================================
--- gnunet/contrib/defaults.conf 2010-08-25 13:52:09 UTC (rev 12733)
+++ gnunet/contrib/defaults.conf 2010-08-25 19:10:18 UTC (rev 12734)
@@ -200,6 +200,9 @@
[datastore-sqlite]
FILENAME = $SERVICEHOME/datastore/sqlite.db
+[datastore-postgres]
+CONFIG = connect_timeout=10;dbname=gnunet
+
[datastore-mysql]
DATABASE = gnunet
# CONFIG = ~/.my.cnf
Added: gnunet/doc/README.mysql
===================================================================
--- gnunet/doc/README.mysql (rev 0)
+++ gnunet/doc/README.mysql 2010-08-25 19:10:18 UTC (rev 12734)
@@ -0,0 +1,95 @@
+How to setup the MySQL database for GNUnet.
+
+NOTE: This db module does NOT work with mysql before 4.1 since we need
+prepared statements. We are generally testing the code against MySQL
+5.0 at this point.
+
+HIGHLIGHTS
+
+Pros
+ + On up-to-date hardware where mysql can be used comfortably, this
+ module will have better performance than the other db choices
+ (according to our tests).
+ + Its often possible to recover the mysql database from internal
+ inconsistencies. The other db choices do not support repair
+ (gnunet-check cannot fix problems internal to the dbmgr!).
+ For example, we have seen several cases where power failure
+ has ruined a gdbm database beyond repair.
+ + much faster (for one of the key benchmarks -- content migration
+ -- we have measure mysql taking 2s for an operation where
+ sqlite takes 150s).
+Cons
+ - Memory usage (Comment: "I have 1G and it never caused me trouble")
+ - Manual setup
+
+MANUAL SETUP INSTRUCTIONS
+
+ 1) in /etc/gnunet.conf, set
+ sqstore = "sqstore_mysql"
+
+ 2) Then access mysql as root,
+ $ mysql -u root -p
+ and do the following. [You should replace $USER with the username
+ that will be running the gnunetd process].
+
+ CREATE DATABASE gnunet;
+ GRANT select,insert,update,delete,create,alter,drop,create temporary
tables
+ ON gnunet.* TO address@hidden;
+ SET PASSWORD FOR address@hidden('$the_password_you_like');
+ FLUSH PRIVILEGES;
+
+ 3) In the $HOME directory of $USER, create a ".my.cnf" file
+ with the following lines
+
+ [client]
+ user=$USER
+ password=$the_password_you_like
+
+ Thats it. Note that .my.cnf file is a security risk unless its on
+ a safe partition etc. The $HOME/.my.cnf can of course be a symbolic
+ link. Even greater security risk can be achieved by setting no
+ password for $USER. Luckily $USER has only priviledges to mess
+ up GNUnet's tables, nothing else (unless you give him more,
+ of course).
+
+ 4) Still, perhaps you should briefly try if the DB connection
+ works. First, login as $USER. Then use,
+
+ $ mysql -u $USER
+ mysql> use gnunet;
+
+ If you get the message "Database changed" it probably works.
+
+ [If you get "ERROR 2002: Can't connect to local MySQL server
+ through socket '/tmp/mysql.sock' (2)" it may be resolvable by
+ "ln -s /var/run/mysqld/mysqld.sock /tmp/mysql.sock"
+ so there may be some additional trouble depending on your mysql setup.]
+
+ 5) If you want to run the testcases, you must create a second
+ database "gnunetcheck" with the same username and password.
+ This database will then be used for testing ("make check").
+
+
+REPAIRING TABLES
+
+- Its probably healthy to check your tables for inconsistencies
+ every now and then, especially after system crashes.
+- If you get odd SEGVs on gnunetd startup, it might be that the mysql
+ databases have been corrupted.
+- The tables can be verified/fixed in two ways;
+ 1) by shutting down mysqld (mandatory!) and running
+ # myisamchk -r *.MYI
+ in /var/lib/mysql/gnunet/ (or wherever the tables are stored).
+ Another repair command is "mysqlcheck". The usable command
+ may depend on your mysql build/version. Or,
+ 2) by executing
+ mysql> REPAIR TABLE gn090;
+
+
+PROBLEMS?
+
+If you have problems related to the mysql module, your best friend is
+probably the mysql manual. The first thing to check is that mysql is
+basically operational, that you can connect to it, create tables,
+issue queries etc.
+
Added: gnunet/doc/README.postgres
===================================================================
--- gnunet/doc/README.postgres (rev 0)
+++ gnunet/doc/README.postgres 2010-08-25 19:10:18 UTC (rev 12734)
@@ -0,0 +1,49 @@
+How to setup the Postgres database for GNUnet.
+
+NOTE: This db module was developed for Postgres 8.3. I have no
+idea what the minimum version that we require is exactly.
+
+HIGHLIGHTS
+
+Pros
+ + Easier to setup than MySQL
+ + Real database
+Cons
+ - Quite slow
+ - Still some setup
+
+MANUAL SETUP INSTRUCTIONS
+
+ 1) in /etc/gnunet.conf, set
+ sqstore = "sqstore_postgres"
+
+ 2) Then access postgres to create a user; I had to do this to get
+ access and create a user:
+ # su - postgres
+ $ createuser
+ At this point, use the name of the user running gnunet
+ for the role, do not set it to superuser, allow the creation
+ of databases.
+
+ 3) As that user, create a database (or two):
+ $ createdb gnunet
+ $ createdb gnunetcheck # this way you can run "make check"
+
+ Thats it.
+
+ 4) Still, perhaps you should briefly try if the DB connection
+ works. First, login as the user who will run gnunetd. Then use,
+
+ $ psql gnunet # or gnunetcheck
+ gnunet=> \dt
+
+ If, after you have started gnunetd at least once, you get a
+ gn090 table here, it probably works.
+
+PROBLEMS?
+
+If you have problems related to the postgres module, your best friend
+is probably the postgres manual. The first thing to check is that
+postgres is basically operational, that you can connect to it, create
+tables, issue queries etc. (see step 4 above for details).
+
Property changes on: gnunet/src/datastore
___________________________________________________________________
Modified: svn:ignore
- test_datastore_api_sqlite
test_datastore_api_mysql
test_datastore_api_management_sqlite
test_datastore_api_management_mysql
perf_plugin_datastore_sqlite
perf_plugin_datastore_mysql
perf_datastore_api_sqlite
perf_datastore_api_mysql
plugin_datastore_sqlite.gcda
datastore_api.gcda
test_datastore_api_management.gcno
test_datastore_api_management.gcda
test_datastore_api_management
test_datastore_api.gcno
test_datastore_api.gcda
plugin_datastore_template.gcno
plugin_datastore_sqlite.gcno
perf_plugin_datastore.gcno
perf_plugin_datastore.gcda
perf_datastore_api.gcno
perf_datastore_api.gcda
gnunet-service-datastore.gcda
datastore_api.gcno
gnunet-service-datastore.gcno
perf_plugin_datastore
test_datastore_api
perf_datastore_api_iterators
perf_datastore_api
Makefile.in
Makefile
gnunet-service-datastore
.deps
+ test_datastore_api_postgres
test_datastore_api_management_postgres
perf_plugin_datastore_postgres
perf_datastore_api_postgres
test_datastore_api_sqlite
test_datastore_api_mysql
test_datastore_api_management_sqlite
test_datastore_api_management_mysql
perf_plugin_datastore_sqlite
perf_plugin_datastore_mysql
perf_datastore_api_sqlite
perf_datastore_api_mysql
plugin_datastore_sqlite.gcda
datastore_api.gcda
test_datastore_api_management.gcno
test_datastore_api_management.gcda
test_datastore_api_management
test_datastore_api.gcno
test_datastore_api.gcda
plugin_datastore_template.gcno
plugin_datastore_sqlite.gcno
perf_plugin_datastore.gcno
perf_plugin_datastore.gcda
perf_datastore_api.gcno
perf_datastore_api.gcda
gnunet-service-datastore.gcda
datastore_api.gcno
gnunet-service-datastore.gcno
perf_plugin_datastore
test_datastore_api
perf_datastore_api_iterators
perf_datastore_api
Makefile.in
Makefile
gnunet-service-datastore
.deps
Modified: gnunet/src/datastore/Makefile.am
===================================================================
--- gnunet/src/datastore/Makefile.am 2010-08-25 13:52:09 UTC (rev 12733)
+++ gnunet/src/datastore/Makefile.am 2010-08-25 19:10:18 UTC (rev 12734)
@@ -51,10 +51,19 @@
perf_datastore_api_sqlite \
perf_plugin_datastore_sqlite
endif
+if HAVE_POSTGRES
+ POSTGRES_PLUGIN = libgnunet_plugin_datastore_postgres.la
+ POSTGRES_TESTS = \
+ test_datastore_api_postgres \
+ test_datastore_api_management_postgres \
+ perf_datastore_api_postgres \
+ perf_plugin_datastore_postgres
+endif
plugin_LTLIBRARIES = \
$(SQLITE_PLUGIN) \
$(MYSQL_PLUGIN) \
+ $(POSTGRES_PLUGIN) \
libgnunet_plugin_datastore_template.la
@@ -77,7 +86,17 @@
libgnunet_plugin_datastore_mysql_la_CPPFLAGS = \
$(MYSQL_CPPFLAGS)
+libgnunet_plugin_datastore_postgres_la_SOURCES = \
+ plugin_datastore_postgres.c
+libgnunet_plugin_datastore_postgres_la_LIBADD = \
+ $(top_builddir)/src/statistics/libgnunetstatistics.la \
+ $(top_builddir)/src/util/libgnunetutil.la $(XLIBS) -lpq
+libgnunet_plugin_datastore_postgres_la_LDFLAGS = \
+ $(GN_PLUGIN_LDFLAGS) $(POSTGRES_LDFLAGS) -lpq
+libgnunet_plugin_datastore_postgres_la_CPPFLAGS = \
+ $(POSTGRES_CPPFLAGS)
+
libgnunet_plugin_datastore_template_la_SOURCES = \
plugin_datastore_template.c
libgnunet_plugin_datastore_template_la_LIBADD = \
@@ -87,7 +106,8 @@
check_PROGRAMS = \
$(SQLITE_TESTS) \
- $(MYSQL_TESTS)
+ $(MYSQL_TESTS) \
+ $(POSTGRES_TESTS)
if !DISABLE_TEST_RUN
TESTS = $(check_PROGRAMS)
@@ -141,8 +161,34 @@
$(top_builddir)/src/util/libgnunetutil.la
+test_datastore_api_postgres_SOURCES = \
+ test_datastore_api.c
+test_datastore_api_postgres_LDADD = \
+ $(top_builddir)/src/datastore/libgnunetdatastore.la \
+ $(top_builddir)/src/util/libgnunetutil.la
+
+test_datastore_api_management_postgres_SOURCES = \
+ test_datastore_api_management.c
+test_datastore_api_management_postgres_LDADD = \
+ $(top_builddir)/src/datastore/libgnunetdatastore.la \
+ $(top_builddir)/src/util/libgnunetutil.la
+
+perf_datastore_api_postgres_SOURCES = \
+ perf_datastore_api.c
+perf_datastore_api_postgres_LDADD = \
+ $(top_builddir)/src/datastore/libgnunetdatastore.la \
+ $(top_builddir)/src/util/libgnunetutil.la
+
+perf_plugin_datastore_postgres_SOURCES = \
+ perf_plugin_datastore.c
+perf_plugin_datastore_postgres_LDADD = \
+ $(top_builddir)/src/util/libgnunetutil.la
+
+
EXTRA_DIST = \
test_datastore_api_data_sqlite.conf \
perf_plugin_datastore_data_sqlite.conf \
test_datastore_api_data_mysql.conf \
- perf_plugin_datastore_data_mysql.conf
+ perf_plugin_datastore_data_mysql.conf \
+ test_datastore_api_data_postgres.conf \
+ perf_plugin_datastore_data_postgres.conf
Added: gnunet/src/datastore/perf_plugin_datastore_data_postgres.conf
===================================================================
--- gnunet/src/datastore/perf_plugin_datastore_data_postgres.conf
(rev 0)
+++ gnunet/src/datastore/perf_plugin_datastore_data_postgres.conf
2010-08-25 19:10:18 UTC (rev 12734)
@@ -0,0 +1,33 @@
+[PATHS]
+SERVICEHOME = /tmp/perf-gnunet-datastore-postgres/
+
+[datastore]
+PORT = 22654
+HOSTNAME = localhost
+HOME = $SERVICEHOME
+CONFIG = $DEFAULTCONFIG
+BINARY = gnunet-service-datastore
+ACCEPT_FROM = 127.0.0.1;
+ACCEPT_FROM6 = ::1;
+QUOTA = 1000000
+BLOOMFILTER = $SERVICEHOME/fs/bloomfilter
+DATABASE = postgres
+# USERNAME =
+# MAXBUF =
+# TIMEOUT =
+# DISABLEV6 =
+# BINDTO =
+# REJECT_FROM =
+# REJECT_FROM6 =
+# PREFIX =
+
+
+[dht]
+AUTOSTART = NO
+
+[datastore-postgres]
+CONFIG = dbname=gnunetcheck
+# USER =
+# PASSWORD =
+# HOST =
+# PORT =
Added: gnunet/src/datastore/plugin_datastore_postgres.c
===================================================================
--- gnunet/src/datastore/plugin_datastore_postgres.c
(rev 0)
+++ gnunet/src/datastore/plugin_datastore_postgres.c 2010-08-25 19:10:18 UTC
(rev 12734)
@@ -0,0 +1,1190 @@
+/*
+ This file is part of GNUnet
+ (C) 2009, 2010 Christian Grothoff (and other contributing authors)
+
+ GNUnet is free software; you can redistribute it and/or modify
+ it under the terms of the GNU General Public License as published
+ by the Free Software Foundation; either version 3, or (at your
+ option) any later version.
+
+ GNUnet is distributed in the hope that it will be useful, but
+ WITHOUT ANY WARRANTY; without even the implied warranty of
+ MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the GNU
+ General Public License for more details.
+
+ You should have received a copy of the GNU General Public License
+ along with GNUnet; see the file COPYING. If not, write to the
+ Free Software Foundation, Inc., 59 Temple Place - Suite 330,
+ Boston, MA 02111-1307, USA.
+*/
+
+/**
+ * @file datastore/plugin_datastore_postgres.c
+ * @brief postgres-based datastore backend
+ * @author Christian Grothoff
+ */
+
+#include "platform.h"
+#include "plugin_datastore.h"
+#include <postgresql/libpq-fe.h>
+
+#define DEBUG_POSTGRES GNUNET_NO
+
+#define SELECT_IT_LOW_PRIORITY "(SELECT size, type, prio, anonLevel, expire,
hash, value, oid FROM gn080 "\
+ "WHERE (prio = $1 AND oid > $2) "
\
+ "ORDER BY prio ASC,oid ASC LIMIT 1) "\
+ "UNION "\
+ "(SELECT size, type, prio, anonLevel, expire,
hash, value, oid FROM gn080 "\
+ "WHERE (prio > $1 AND oid != $2)"\
+ "ORDER BY prio ASC,oid ASC LIMIT 1)"\
+ "ORDER BY prio ASC,oid ASC LIMIT 1"
+
+#define SELECT_IT_NON_ANONYMOUS "(SELECT size, type, prio, anonLevel, expire,
hash, value, oid FROM gn080 "\
+ "WHERE (prio = $1 AND oid < $2)"\
+ " AND anonLevel=0 ORDER BY prio DESC,oid DESC
LIMIT 1) "\
+ "UNION "\
+ "(SELECT size, type, prio, anonLevel, expire,
hash, value, oid FROM gn080 "\
+ "WHERE (prio < $1 AND oid != $2)"\
+ " AND anonLevel=0 ORDER BY prio DESC,oid DESC
LIMIT 1) "\
+ "ORDER BY prio DESC,oid DESC LIMIT 1"
+
+#define SELECT_IT_EXPIRATION_TIME "(SELECT size, type, prio, anonLevel,
expire, hash, value, oid FROM gn080 "\
+ "WHERE (expire = $1 AND oid > $2) "\
+ "ORDER BY expire ASC,oid ASC LIMIT 1) "\
+ "UNION "\
+ "(SELECT size, type, prio, anonLevel,
expire, hash, value, oid FROM gn080 "\
+ "WHERE (expire > $1 AND oid != $2) "
\
+ "ORDER BY expire ASC,oid ASC LIMIT 1)"\
+ "ORDER BY expire ASC,oid ASC LIMIT 1"
+
+
+#define SELECT_IT_MIGRATION_ORDER "(SELECT size, type, prio, anonLevel,
expire, hash, value, oid FROM gn080 "\
+ "WHERE (expire = $1 AND oid < $2)"\
+ " AND expire > $3 AND type!=3"\
+ " ORDER BY expire DESC,oid DESC LIMIT 1) "\
+ "UNION "\
+ "(SELECT size, type, prio, anonLevel,
expire, hash, value, oid FROM gn080 "\
+ "WHERE (expire < $1 AND oid != $2)"
\
+ " AND expire > $3 AND type!=3"\
+ " ORDER BY expire DESC,oid DESC LIMIT 1)"\
+ "ORDER BY expire DESC,oid DESC LIMIT 1"
+
+/**
+ * After how many ms "busy" should a DB operation fail for good?
+ * A low value makes sure that we are more responsive to requests
+ * (especially PUTs). A high value guarantees a higher success
+ * rate (SELECTs in iterate can take several seconds despite LIMIT=1).
+ *
+ * The default value of 1s should ensure that users do not experience
+ * huge latencies while at the same time allowing operations to succeed
+ * with reasonable probability.
+ */
+#define BUSY_TIMEOUT GNUNET_TIME_UNIT_SECONDS
+
+
+struct NextRequestClosure
+{
+ struct Plugin *plugin;
+ PluginIterator iter;
+ void *iter_cls;
+ const char *paramValues[5];
+ const char *pname;
+ int paramLengths[5];
+ int nparams; // nparams
+ struct GNUNET_TIME_Absolute now;
+ GNUNET_HashCode key;
+ GNUNET_HashCode vhash;
+ long long count;
+ long long off;
+ long long limit_off;
+ unsigned long long total;
+ unsigned long long last_expire;
+ unsigned int last_rowid; // last_rowid
+ unsigned int last_prio;
+ enum GNUNET_BLOCK_Type type;
+ int end_it;
+};
+
+
+/**
+ * Context for all functions in this plugin.
+ */
+struct Plugin
+{
+ /**
+ * Our execution environment.
+ */
+ struct GNUNET_DATASTORE_PluginEnvironment *env;
+
+ /**
+ * Native Postgres database handle.
+ */
+ PGconn *dbh;
+
+ /**
+ * Closure of the 'next_task' (must be freed if 'next_task' is cancelled).
+ */
+ struct NextRequestClosure *next_task_nc;
+
+ /**
+ * Pending task with scheduler for running the next request.
+ */
+ GNUNET_SCHEDULER_TaskIdentifier next_task;
+
+ unsigned long long payload;
+
+ unsigned int lastSync;
+
+};
+
+
+/**
+ * Check if the result obtained from Postgres has
+ * the desired status code. If not, log an error, clear the
+ * result and return GNUNET_SYSERR.
+ *
+ * @return GNUNET_OK if the result is acceptable
+ */
+static int
+check_result (struct Plugin *plugin,
+ PGresult * ret,
+ int expected_status,
+ const char *command, const char *args, int line)
+{
+ if (ret == NULL)
+ {
+ GNUNET_log_from (GNUNET_ERROR_TYPE_ERROR | GNUNET_ERROR_TYPE_BULK,
+ "datastore-postgres",
+ "Postgres failed to allocate result for `%s:%s' at %d\n",
+ command, args, line);
+ return GNUNET_SYSERR;
+ }
+ if (PQresultStatus (ret) != expected_status)
+ {
+ GNUNET_log_from (GNUNET_ERROR_TYPE_ERROR | GNUNET_ERROR_TYPE_BULK,
+ "datastore-postgres",
+ _("`%s:%s' failed at %s:%d with error: %s"),
+ command, args, __FILE__, line, PQerrorMessage
(plugin->dbh));
+ PQclear (ret);
+ return GNUNET_SYSERR;
+ }
+ return GNUNET_OK;
+}
+
+/**
+ * Run simple SQL statement (without results).
+ */
+static int
+pq_exec (struct Plugin *plugin,
+ const char *sql, int line)
+{
+ PGresult *ret;
+ ret = PQexec (plugin->dbh, sql);
+ if (GNUNET_OK != check_result (plugin,
+ ret,
+ PGRES_COMMAND_OK, "PQexec", sql, line))
+ return GNUNET_SYSERR;
+ PQclear (ret);
+ return GNUNET_OK;
+}
+
+/**
+ * Prepare SQL statement.
+ */
+static int
+pq_prepare (struct Plugin *plugin,
+ const char *name, const char *sql, int nparms, int line)
+{
+ PGresult *ret;
+ ret = PQprepare (plugin->dbh, name, sql, nparms, NULL);
+ if (GNUNET_OK !=
+ check_result (plugin,
+ ret, PGRES_COMMAND_OK, "PQprepare", sql, line))
+ return GNUNET_SYSERR;
+ PQclear (ret);
+ return GNUNET_OK;
+}
+
+/**
+ * @brief Get a database handle
+ * @return GNUNET_OK on success, GNUNET_SYSERR on error
+ */
+static int
+init_connection (struct Plugin *plugin)
+{
+ char *conninfo;
+ PGresult *ret;
+
+ /* Open database and precompile statements */
+ conninfo = NULL;
+ GNUNET_CONFIGURATION_get_value_string (plugin->env->cfg,
+ "datastore-postgres",
+ "CONFIG",
+ &conninfo);
+ plugin->dbh = PQconnectdb (conninfo == NULL ? "" : conninfo);
+ GNUNET_free_non_null (conninfo);
+ if (NULL == plugin->dbh)
+ {
+ /* FIXME: warn about out-of-memory? */
+ return GNUNET_SYSERR;
+ }
+ if (PQstatus (plugin->dbh) != CONNECTION_OK)
+ {
+ GNUNET_log_from (GNUNET_ERROR_TYPE_ERROR,
+ "postgres",
+ _("Unable to initialize Postgres: %s"),
+ PQerrorMessage (plugin->dbh));
+ PQfinish (plugin->dbh);
+ plugin->dbh = NULL;
+ return GNUNET_SYSERR;
+ }
+ ret = PQexec (plugin->dbh,
+ "CREATE TABLE gn080 ("
+ " size INTEGER NOT NULL DEFAULT 0,"
+ " type INTEGER NOT NULL DEFAULT 0,"
+ " prio INTEGER NOT NULL DEFAULT 0,"
+ " anonLevel INTEGER NOT NULL DEFAULT 0,"
+ " expire BIGINT NOT NULL DEFAULT 0,"
+ " hash BYTEA NOT NULL DEFAULT '',"
+ " vhash BYTEA NOT NULL DEFAULT '',"
+ " value BYTEA NOT NULL DEFAULT '')" "WITH OIDS");
+ if ( (ret == NULL) ||
+ ( (PQresultStatus (ret) != PGRES_COMMAND_OK) &&
+ (0 != strcmp ("42P07", /* duplicate table */
+ PQresultErrorField
+ (ret,
+ PG_DIAG_SQLSTATE)))))
+ {
+ check_result (plugin,
+ ret, PGRES_COMMAND_OK, "CREATE TABLE", "gn080", __LINE__);
+ PQfinish (plugin->dbh);
+ plugin->dbh = NULL;
+ return GNUNET_SYSERR;
+ }
+ if (PQresultStatus (ret) == PGRES_COMMAND_OK)
+ {
+ if ((GNUNET_OK !=
+ pq_exec (plugin, "CREATE INDEX idx_hash ON gn080 (hash)",
__LINE__)) ||
+ (GNUNET_OK !=
+ pq_exec (plugin, "CREATE INDEX idx_hash_vhash ON gn080
(hash,vhash)",
+ __LINE__))
+ || (GNUNET_OK !=
+ pq_exec (plugin, "CREATE INDEX idx_prio ON gn080 (prio)",
__LINE__))
+ || (GNUNET_OK !=
+ pq_exec (plugin, "CREATE INDEX idx_expire ON gn080 (expire)",
__LINE__))
+ || (GNUNET_OK !=
+ pq_exec (plugin, "CREATE INDEX idx_comb3 ON gn080
(prio,anonLevel)",
+ __LINE__))
+ || (GNUNET_OK !=
+ pq_exec
+ (plugin, "CREATE INDEX idx_comb4 ON gn080 (prio,hash,anonLevel)",
+ __LINE__))
+ || (GNUNET_OK !=
+ pq_exec (plugin, "CREATE INDEX idx_comb7 ON gn080 (expire,hash)",
+ __LINE__)))
+ {
+ PQclear (ret);
+ PQfinish (plugin->dbh);
+ plugin->dbh = NULL;
+ return GNUNET_SYSERR;
+ }
+ }
+ PQclear (ret);
+#if 1
+ ret = PQexec (plugin->dbh,
+ "ALTER TABLE gn080 ALTER value SET STORAGE EXTERNAL");
+ if ( (ret == NULL) ||
+ ((PQresultStatus (ret) != PGRES_COMMAND_OK) ) )
+ {
+ check_result (plugin,
+ ret, PGRES_COMMAND_OK, "ALTER TABLE", "gn080", __LINE__);
+ PQfinish (plugin->dbh);
+ plugin->dbh = NULL;
+ return GNUNET_SYSERR;
+ }
+ ret = PQexec (plugin->dbh,
+ "ALTER TABLE gn080 ALTER hash SET STORAGE PLAIN");
+ if ( (ret == NULL) ||
+ ((PQresultStatus (ret) != PGRES_COMMAND_OK) ) )
+ {
+ check_result (plugin,
+ ret, PGRES_COMMAND_OK, "ALTER TABLE", "gn080", __LINE__);
+ PQfinish (plugin->dbh);
+ plugin->dbh = NULL;
+ return GNUNET_SYSERR;
+ }
+ ret = PQexec (plugin->dbh,
+ "ALTER TABLE gn080 ALTER vhash SET STORAGE PLAIN");
+ if ( (ret == NULL) ||
+ ((PQresultStatus (ret) != PGRES_COMMAND_OK) ) )
+ {
+ check_result (plugin,
+ ret, PGRES_COMMAND_OK, "ALTER TABLE", "gn080", __LINE__);
+ PQfinish (plugin->dbh);
+ plugin->dbh = NULL;
+ return GNUNET_SYSERR;
+ }
+#endif
+ if ((GNUNET_OK !=
+ pq_prepare (plugin,
+ "getvt",
+ "SELECT size, type, prio, anonLevel, expire, hash, value,
oid FROM gn080 "
+ "WHERE hash=$1 AND vhash=$2 AND type=$3 "
+ "AND oid >= $4 ORDER BY oid ASC LIMIT 1 OFFSET $5",
+ 5,
+ __LINE__)) ||
+ (GNUNET_OK !=
+ pq_prepare (plugin,
+ "gett",
+ "SELECT size, type, prio, anonLevel, expire, hash, value,
oid FROM gn080 "
+ "WHERE hash=$1 AND type=$2"
+ "AND oid >= $3 ORDER BY oid ASC LIMIT 1 OFFSET $4",
+ 4,
+ __LINE__)) ||
+ (GNUNET_OK !=
+ pq_prepare (plugin,
+ "getv",
+ "SELECT size, type, prio, anonLevel, expire, hash, value,
oid FROM gn080 "
+ "WHERE hash=$1 AND vhash=$2"
+ "AND oid >= $3 ORDER BY oid ASC LIMIT 1 OFFSET $4",
+ 4,
+ __LINE__)) ||
+ (GNUNET_OK !=
+ pq_prepare (plugin,
+ "get",
+ "SELECT size, type, prio, anonLevel, expire, hash, value,
oid FROM gn080 "
+ "WHERE hash=$1"
+ "AND oid >= $2 ORDER BY oid ASC LIMIT 1 OFFSET $3",
+ 3,
+ __LINE__)) ||
+ (GNUNET_OK !=
+ pq_prepare (plugin,
+ "put",
+ "INSERT INTO gn080 (size, type, prio, anonLevel, expire,
hash, vhash, value) "
+ "VALUES ($1, $2, $3, $4, $5, $6, $7, $8)",
+ 8,
+ __LINE__)) ||
+ (GNUNET_OK !=
+ pq_prepare (plugin,
+ "update",
+ "UPDATE gn080 SET prio = prio + $1, expire = CASE WHEN
expire < $2 THEN $2 ELSE expire END "
+ "WHERE oid = $3",
+ 3,
+ __LINE__)) ||
+ (GNUNET_OK !=
+ pq_prepare (plugin,
+ "select_low_priority",
+ SELECT_IT_LOW_PRIORITY,
+ 2,
+ __LINE__)) ||
+ (GNUNET_OK !=
+ pq_prepare (plugin,
+ "select_non_anonymous",
+ SELECT_IT_NON_ANONYMOUS,
+ 2,
+ __LINE__)) ||
+ (GNUNET_OK !=
+ pq_prepare (plugin,
+ "select_expiration_time",
+ SELECT_IT_EXPIRATION_TIME,
+ 2,
+ __LINE__)) ||
+ (GNUNET_OK !=
+ pq_prepare (plugin,
+ "select_migration_order",
+ SELECT_IT_MIGRATION_ORDER,
+ 3,
+ __LINE__)) ||
+ (GNUNET_OK !=
+ pq_prepare (plugin,
+ "delrow",
+ "DELETE FROM gn080 " "WHERE oid=$1", 1, __LINE__)))
+ {
+ PQfinish (plugin->dbh);
+ plugin->dbh = NULL;
+ return GNUNET_SYSERR;
+ }
+ return GNUNET_OK;
+}
+
+
+/**
+ * Delete the row identified by the given rowid (qid
+ * in postgres).
+ *
+ * @return GNUNET_OK on success
+ */
+static int
+delete_by_rowid (struct Plugin *plugin,
+ unsigned int rowid)
+{
+ const char *paramValues[] = { (const char *) &rowid };
+ int paramLengths[] = { sizeof (rowid) };
+ const int paramFormats[] = { 1 };
+ PGresult *ret;
+
+ ret = PQexecPrepared (plugin->dbh,
+ "delrow",
+ 1, paramValues, paramLengths, paramFormats, 1);
+ if (GNUNET_OK !=
+ check_result (plugin,
+ ret, PGRES_COMMAND_OK, "PQexecPrepared", "delrow",
+ __LINE__))
+ {
+ return GNUNET_SYSERR;
+ }
+ PQclear (ret);
+ return GNUNET_OK;
+}
+
+
+/**
+ * Get an estimate of how much space the database is
+ * currently using.
+ *
+ * @param cls our "struct Plugin*"
+ * @return number of bytes used on disk
+ */
+static unsigned long long
+postgres_plugin_get_size (void *cls)
+{
+ struct Plugin *plugin = cls;
+ double ret;
+
+ ret = plugin->payload;
+ return (unsigned long long) (ret * 1.00);
+ /* benchmarking shows XX% overhead */
+}
+
+
+/**
+ * Store an item in the datastore.
+ *
+ * @param cls closure
+ * @param key key for the item
+ * @param size number of bytes in data
+ * @param data content stored
+ * @param type type of the content
+ * @param priority priority of the content
+ * @param anonymity anonymity-level for the content
+ * @param expiration expiration time for the content
+ * @param msg set to error message
+ * @return GNUNET_OK on success
+ */
+static int
+postgres_plugin_put (void *cls,
+ const GNUNET_HashCode * key,
+ uint32_t size,
+ const void *data,
+ enum GNUNET_BLOCK_Type type,
+ uint32_t priority,
+ uint32_t anonymity,
+ struct GNUNET_TIME_Absolute expiration,
+ char **msg)
+{
+ struct Plugin *plugin = cls;
+ GNUNET_HashCode vhash;
+ PGresult *ret;
+ const char *paramValues[] = {
+ (const char *) &size,
+ (const char *) &type,
+ (const char *) &priority,
+ (const char *) &anonymity,
+ (const char *) &expiration.value,
+ (const char *) key,
+ (const char *) &vhash,
+ (const char *) data
+ };
+ int paramLengths[] = {
+ sizeof (size),
+ sizeof (type),
+ sizeof (priority),
+ sizeof (anonymity),
+ sizeof (expiration.value),
+ sizeof (GNUNET_HashCode),
+ sizeof (GNUNET_HashCode),
+ size
+ };
+ const int paramFormats[] = { 1, 1, 1, 1, 1, 1, 1, 1 };
+
+ GNUNET_CRYPTO_hash (data, size, &vhash);
+ ret = PQexecPrepared (plugin->dbh,
+ "put", 8, paramValues, paramLengths, paramFormats, 1);
+ if (GNUNET_OK != check_result (plugin, ret,
+ PGRES_COMMAND_OK,
+ "PQexecPrepared", "put", __LINE__))
+ return GNUNET_SYSERR;
+ PQclear (ret);
+ plugin->payload += size;
+ return GNUNET_OK;
+}
+
+/**
+ * Function invoked on behalf of a "PluginIterator"
+ * asking the database plugin to call the iterator
+ * with the next item.
+ *
+ * @param cls the 'struct NextRequestClosure'
+ * @param tc scheduler context
+ */
+static void
+postgres_next_request_cont (void *next_cls,
+ const struct GNUNET_SCHEDULER_TaskContext *tc)
+{
+ struct NextRequestClosure *nrc = next_cls;
+ struct Plugin *plugin = nrc->plugin;
+ const int paramFormats[] = { 1, 1, 1, 1, 1 };
+ int iret;
+ PGresult *res;
+ enum GNUNET_BLOCK_Type type;
+ unsigned int anonymity;
+ unsigned int priority;
+ unsigned int size;
+ unsigned int rowid;
+ struct GNUNET_TIME_Absolute expiration_time;
+ GNUNET_HashCode key;
+
+ plugin->next_task = GNUNET_SCHEDULER_NO_TASK;
+ plugin->next_task_nc = NULL;
+ if (GNUNET_YES == nrc->end_it)
+ {
+ nrc->iter (nrc->iter_cls,
+ NULL, NULL, 0, NULL, 0, 0, 0,
+ GNUNET_TIME_UNIT_ZERO_ABS, 0);
+ GNUNET_free (nrc);
+ return;
+ }
+
+
+ if (nrc->count == 0)
+ nrc->limit_off = nrc->off;
+ else
+ nrc->limit_off = 0;
+
+ res = PQexecPrepared (plugin->dbh,
+ nrc->pname,
+ nrc->nparams,
+ nrc->paramValues,
+ nrc->paramLengths,
+ paramFormats, 1);
+ if (GNUNET_OK != check_result (plugin,
+ res,
+ PGRES_TUPLES_OK,
+ "PQexecPrepared",
+ nrc->pname,
+ __LINE__))
+ {
+ nrc->iter (nrc->iter_cls,
+ NULL, NULL, 0, NULL, 0, 0, 0,
+ GNUNET_TIME_UNIT_ZERO_ABS, 0);
+ GNUNET_free (nrc);
+ return;
+ }
+
+ if (0 == PQntuples (res))
+ {
+ /* no result */
+ nrc->iter (nrc->iter_cls,
+ NULL, NULL, 0, NULL, 0, 0, 0,
+ GNUNET_TIME_UNIT_ZERO_ABS, 0);
+ GNUNET_free (nrc);
+ return;
+ }
+ if ((1 != PQntuples (res)) ||
+ (8 != PQnfields (res)) ||
+ (sizeof (unsigned int) != PQfsize (res, 0)) ||
+ (sizeof (unsigned int) != PQfsize (res, 7)))
+ {
+ GNUNET_break (0);
+ nrc->iter (nrc->iter_cls,
+ NULL, NULL, 0, NULL, 0, 0, 0,
+ GNUNET_TIME_UNIT_ZERO_ABS, 0);
+ GNUNET_free (nrc);
+ return;
+ }
+ rowid = *(unsigned int *) PQgetvalue (res, 0, 7);
+ size = *(unsigned int *) PQgetvalue (res, 0, 0);
+ if ((sizeof (unsigned int) != PQfsize (res, 1)) ||
+ (sizeof (unsigned int) != PQfsize (res, 2)) ||
+ (sizeof (unsigned int) != PQfsize (res, 3)) ||
+ (sizeof (unsigned long long) != PQfsize (res, 4)) ||
+ (sizeof (GNUNET_HashCode) != PQgetlength (res, 0, 5)) ||
+ (size != PQgetlength (res, 0, 6)))
+ {
+ GNUNET_break (0);
+ delete_by_rowid (plugin, rowid);
+ nrc->iter (nrc->iter_cls,
+ NULL, NULL, 0, NULL, 0, 0, 0,
+ GNUNET_TIME_UNIT_ZERO_ABS, 0);
+ GNUNET_free (nrc);
+ return;
+ }
+
+ type = *(unsigned int *) PQgetvalue (res, 0, 1);
+ priority = *(unsigned int *) PQgetvalue (res, 0, 2);
+ anonymity = *(unsigned int *) PQgetvalue (res, 0, 3);
+ expiration_time.value = *(unsigned long long *) PQgetvalue (res, 0, 4);
+ size = PQgetlength (res, 0, 6);
+ memcpy (&key, PQgetvalue (res, 0, 5), sizeof (GNUNET_HashCode));
+
+ nrc->last_prio = priority;
+ nrc->last_expire = expiration_time.value;
+ nrc->last_rowid = rowid + 1;
+ nrc->count++;
+ iret = nrc->iter (nrc->iter_cls,
+ nrc,
+ &key,
+ size,
+ PQgetvalue (res, 0, 6),
+ (enum GNUNET_BLOCK_Type) type,
+ priority,
+ anonymity,
+ expiration_time,
+ rowid);
+ if (iret == GNUNET_SYSERR)
+ return;
+ if (iret == GNUNET_NO)
+ {
+ plugin->payload -= size;
+ delete_by_rowid (plugin, rowid);
+ }
+}
+
+
+
+/**
+ * Function invoked on behalf of a "PluginIterator"
+ * asking the database plugin to call the iterator
+ * with the next item.
+ *
+ * @param next_cls whatever argument was given
+ * to the PluginIterator as "next_cls".
+ * @param end_it set to GNUNET_YES if we
+ * should terminate the iteration early
+ * (iterator should be still called once more
+ * to signal the end of the iteration).
+ */
+static void
+postgres_plugin_next_request (void *next_cls,
+ int end_it)
+{
+ struct NextRequestClosure *nrc = next_cls;
+
+ if (GNUNET_YES == end_it)
+ nrc->end_it = GNUNET_YES;
+ nrc->plugin->next_task_nc = nrc;
+ nrc->plugin->next_task = GNUNET_SCHEDULER_add_now (nrc->plugin->env->sched,
+
&postgres_next_request_cont,
+ nrc);
+}
+
+
+/**
+ * Update the priority for a particular key in the datastore. If
+ * the expiration time in value is different than the time found in
+ * the datastore, the higher value should be kept. For the
+ * anonymity level, the lower value is to be used. The specified
+ * priority should be added to the existing priority, ignoring the
+ * priority in value.
+ *
+ * Note that it is possible for multiple values to match this put.
+ * In that case, all of the respective values are updated.
+ *
+ * @param cls our "struct Plugin*"
+ * @param uid unique identifier of the datum
+ * @param delta by how much should the priority
+ * change? If priority + delta < 0 the
+ * priority should be set to 0 (never go
+ * negative).
+ * @param expire new expiration time should be the
+ * MAX of any existing expiration time and
+ * this value
+ * @param msg set to error message
+ * @return GNUNET_OK on success
+ */
+static int
+postgres_plugin_update (void *cls,
+ uint64_t uid,
+ int delta, struct GNUNET_TIME_Absolute expire,
+ char **msg)
+{
+ struct Plugin *plugin = cls;
+ unsigned int oid = (unsigned int) uid; /* only 32 bit for postgres */
+ PGresult *ret;
+
+ const char *paramValues[] = {
+ (const char *) &delta,
+ (const char *) &expire.value,
+ (const char *) &oid,
+ };
+ int paramLengths[] = {
+ sizeof (delta),
+ sizeof (expire.value),
+ sizeof (oid),
+ };
+ const int paramFormats[] = { 1, 1, 1 };
+
+ ret = PQexecPrepared (plugin->dbh,
+ "update",
+ 3, paramValues, paramLengths, paramFormats, 1);
+ if (GNUNET_OK != check_result (plugin,
+ ret,
+ PGRES_COMMAND_OK,
+ "PQexecPrepared", "update", __LINE__))
+ return GNUNET_SYSERR;
+ PQclear (ret);
+ return GNUNET_OK;
+}
+
+
+/**
+ * Call a method for each key in the database and
+ * call the callback method on it.
+ *
+ * @param type entries of which type should be considered?
+ * @param iter maybe NULL (to just count); iter
+ * should return GNUNET_SYSERR to abort the
+ * iteration, GNUNET_NO to delete the entry and
+ * continue and GNUNET_OK to continue iterating
+ */
+static void
+postgres_iterate (struct Plugin *plugin,
+ unsigned int type,
+ int is_asc,
+ unsigned int iter_select,
+ PluginIterator iter, void *iter_cls)
+{
+ struct NextRequestClosure *nrc;
+
+ nrc = GNUNET_malloc (sizeof (struct NextRequestClosure));
+ nrc->plugin = plugin;
+ nrc->iter = iter;
+ nrc->iter_cls = iter_cls;
+ if (is_asc)
+ {
+ nrc->last_prio = 0;
+ nrc->last_rowid = 0;
+ nrc->last_expire = 0;
+ }
+ else
+ {
+ nrc->last_prio = 0x7FFFFFFFL;
+ nrc->last_rowid = 0xFFFFFFFF;
+ nrc->last_expire = 0x7FFFFFFFFFFFFFFFLL;
+ }
+ switch (iter_select)
+ {
+ case 0:
+ nrc->pname = "select_low_priority";
+ nrc->nparams = 2;
+ nrc->paramValues[0] = (const char *) &nrc->last_prio;
+ nrc->paramValues[1] = (const char *) &nrc->last_rowid;
+ nrc->paramLengths[0] = sizeof (nrc->last_prio);
+ nrc->paramLengths[1] = sizeof (nrc->last_rowid);
+ break;
+ case 1:
+ nrc->pname = "select_non_anonymous";
+ nrc->nparams = 2;
+ nrc->paramValues[0] = (const char *) &nrc->last_prio;
+ nrc->paramValues[1] = (const char *) &nrc->last_rowid;
+ nrc->paramLengths[0] = sizeof (nrc->last_prio);
+ nrc->paramLengths[1] = sizeof (nrc->last_rowid);
+ break;
+ case 2:
+ nrc->pname = "select_expiration_time";
+ nrc->nparams = 2;
+ nrc->paramValues[0] = (const char *) &nrc->last_expire;
+ nrc->paramValues[1] = (const char *) &nrc->last_rowid;
+ nrc->paramLengths[0] = sizeof (nrc->last_expire);
+ nrc->paramLengths[1] = sizeof (nrc->last_rowid);
+ break;
+ case 3:
+ nrc->pname = "select_migration_order";
+ nrc->nparams = 3;
+ nrc->paramValues[0] = (const char *) &nrc->last_expire;
+ nrc->paramValues[1] = (const char *) &nrc->last_rowid;
+ nrc->paramValues[2] = (const char *) &nrc->now;
+ nrc->paramLengths[0] = sizeof (nrc->last_expire);
+ nrc->paramLengths[1] = sizeof (nrc->last_rowid);
+ nrc->paramLengths[2] = sizeof (nrc->now);
+ break;
+ default:
+ GNUNET_break (0);
+ iter (iter_cls,
+ NULL, NULL, 0, NULL, 0, 0, 0,
+ GNUNET_TIME_UNIT_ZERO_ABS, 0);
+ return;
+ }
+ nrc->now = GNUNET_TIME_absolute_get ();
+ postgres_plugin_next_request (nrc,
+ GNUNET_NO);
+}
+
+
+/**
+ * Select a subset of the items in the datastore and call
+ * the given iterator for each of them.
+ *
+ * @param cls our "struct Plugin*"
+ * @param type entries of which type should be considered?
+ * Use 0 for any type.
+ * @param iter function to call on each matching value;
+ * will be called once with a NULL value at the end
+ * @param iter_cls closure for iter
+ */
+static void
+postgres_plugin_iter_low_priority (void *cls,
+ enum GNUNET_BLOCK_Type type,
+ PluginIterator iter,
+ void *iter_cls)
+{
+ struct Plugin *plugin = cls;
+
+ postgres_iterate (plugin,
+ type,
+ GNUNET_YES, 0,
+ iter, iter_cls);
+}
+
+
+
+
+/**
+ * Iterate over the results for a particular key
+ * in the datastore.
+ *
+ * @param cls closure
+ * @param key maybe NULL (to match all entries)
+ * @param vhash hash of the value, maybe NULL (to
+ * match all values that have the right key).
+ * Note that for DBlocks there is no difference
+ * betwen key and vhash, but for other blocks
+ * there may be!
+ * @param type entries of which type are relevant?
+ * Use 0 for any type.
+ * @param iter function to call on each matching value;
+ * will be called once with a NULL value at the end
+ * @param iter_cls closure for iter
+ */
+static void
+postgres_plugin_get (void *cls,
+ const GNUNET_HashCode * key,
+ const GNUNET_HashCode * vhash,
+ enum GNUNET_BLOCK_Type type,
+ PluginIterator iter, void *iter_cls)
+{
+ struct Plugin *plugin = cls;
+ struct NextRequestClosure *nrc;
+ const int paramFormats[] = { 1, 1, 1, 1, 1 };
+ PGresult *ret;
+
+ if (key == NULL)
+ {
+ postgres_plugin_iter_low_priority (plugin, type,
+ iter, iter_cls);
+ return;
+ }
+ nrc = GNUNET_malloc (sizeof (struct NextRequestClosure));
+ nrc->plugin = plugin;
+ nrc->iter = iter;
+ nrc->iter_cls = iter_cls;
+ nrc->key = *key;
+ if (vhash != NULL)
+ nrc->vhash = *vhash;
+ nrc->paramValues[0] = (const char*) &nrc->key;
+ nrc->paramLengths[0] = sizeof (GNUNET_HashCode);
+ nrc->type = type;
+ if (type != 0)
+ {
+ if (vhash != NULL)
+ {
+ nrc->paramValues[1] = (const char *) &nrc->vhash;
+ nrc->paramLengths[1] = sizeof (nrc->vhash);
+ nrc->paramValues[2] = (const char *) &nrc->type;
+ nrc->paramLengths[2] = sizeof (nrc->type);
+ nrc->paramValues[3] = (const char *) &nrc->last_rowid;
+ nrc->paramLengths[3] = sizeof (nrc->last_rowid);
+ nrc->paramValues[4] = (const char *) &nrc->limit_off;
+ nrc->paramLengths[4] = sizeof (nrc->limit_off);
+ nrc->nparams = 5;
+ nrc->pname = "getvt";
+ ret = PQexecParams (plugin->dbh,
+ "SELECT count(*) FROM gn080 WHERE hash=$1 AND
vhash=$2 AND type=$3",
+ 3,
+ NULL,
+ nrc->paramValues,
+ nrc->paramLengths,
+ paramFormats, 1);
+ }
+ else
+ {
+ nrc->paramValues[1] = (const char *) &nrc->type;
+ nrc->paramLengths[1] = sizeof (nrc->type);
+ nrc->paramValues[2] = (const char *) &nrc->last_rowid;
+ nrc->paramLengths[2] = sizeof (nrc->last_rowid);
+ nrc->paramValues[3] = (const char *) &nrc->limit_off;
+ nrc->paramLengths[3] = sizeof (nrc->limit_off);
+ nrc->nparams = 4;
+ nrc->pname = "gett";
+ ret = PQexecParams (plugin->dbh,
+ "SELECT count(*) FROM gn080 WHERE hash=$1 AND
type=$2",
+ 2,
+ NULL,
+ nrc->paramValues,
+ nrc->paramLengths,
+ paramFormats, 1);
+ }
+ }
+ else
+ {
+ if (vhash != NULL)
+ {
+ nrc->paramValues[1] = (const char *) &nrc->vhash;
+ nrc->paramLengths[1] = sizeof (nrc->vhash);
+ nrc->paramValues[2] = (const char *) &nrc->last_rowid;
+ nrc->paramLengths[2] = sizeof (nrc->last_rowid);
+ nrc->paramValues[3] = (const char *) &nrc->limit_off;
+ nrc->paramLengths[3] = sizeof (nrc->limit_off);
+ nrc->nparams = 4;
+ nrc->pname = "getv";
+ ret = PQexecParams (plugin->dbh,
+ "SELECT count(*) FROM gn080 WHERE hash=$1 AND
vhash=$2",
+ 2,
+ NULL,
+ nrc->paramValues,
+ nrc->paramLengths,
+ paramFormats, 1);
+ }
+ else
+ {
+ nrc->paramValues[1] = (const char *) &nrc->last_rowid;
+ nrc->paramLengths[1] = sizeof (nrc->last_rowid);
+ nrc->paramValues[2] = (const char *) &nrc->limit_off;
+ nrc->paramLengths[2] = sizeof (nrc->limit_off);
+ nrc->nparams = 3;
+ nrc->pname = "get";
+ ret = PQexecParams (plugin->dbh,
+ "SELECT count(*) FROM gn080 WHERE hash=$1",
+ 1,
+ NULL,
+ nrc->paramValues,
+ nrc->paramLengths,
+ paramFormats, 1);
+ }
+ }
+ if (GNUNET_OK != check_result (plugin,
+ ret,
+ PGRES_TUPLES_OK,
+ "PQexecParams",
+ nrc->pname,
+ __LINE__))
+ {
+ iter (iter_cls,
+ NULL, NULL, 0, NULL, 0, 0, 0,
+ GNUNET_TIME_UNIT_ZERO_ABS, 0);
+ return;
+ }
+ if ((PQntuples (ret) != 1) ||
+ (PQnfields (ret) != 1) ||
+ (PQgetlength (ret, 0, 0) != sizeof (unsigned long long)))
+ {
+ GNUNET_break (0);
+ PQclear (ret);
+ iter (iter_cls,
+ NULL, NULL, 0, NULL, 0, 0, 0,
+ GNUNET_TIME_UNIT_ZERO_ABS, 0);
+ return;
+ }
+ nrc->total = GNUNET_ntohll (*(const unsigned long long *) PQgetvalue (ret,
0, 0));
+ PQclear (ret);
+ if (nrc->total == 0)
+ {
+ iter (iter_cls,
+ NULL, NULL, 0, NULL, 0, 0, 0,
+ GNUNET_TIME_UNIT_ZERO_ABS, 0);
+ return;
+ }
+ nrc->off = GNUNET_CRYPTO_random_u64 (GNUNET_CRYPTO_QUALITY_WEAK,
+ nrc->total);
+ postgres_plugin_next_request (nrc,
+ GNUNET_NO);
+}
+
+
+/**
+ * Select a subset of the items in the datastore and call
+ * the given iterator for each of them.
+ *
+ * @param cls our "struct Plugin*"
+ * @param type entries of which type should be considered?
+ * Use 0 for any type.
+ * @param iter function to call on each matching value;
+ * will be called once with a NULL value at the end
+ * @param iter_cls closure for iter
+ */
+static void
+postgres_plugin_iter_zero_anonymity (void *cls,
+ enum GNUNET_BLOCK_Type type,
+ PluginIterator iter,
+ void *iter_cls)
+{
+ struct Plugin *plugin = cls;
+
+ postgres_iterate (plugin,
+ type, GNUNET_NO, 1,
+ iter, iter_cls);
+}
+
+
+/**
+ * Select a subset of the items in the datastore and call
+ * the given iterator for each of them.
+ *
+ * @param cls our "struct Plugin*"
+ * @param type entries of which type should be considered?
+ * Use 0 for any type.
+ * @param iter function to call on each matching value;
+ * will be called once with a NULL value at the end
+ * @param iter_cls closure for iter
+ */
+static void
+postgres_plugin_iter_ascending_expiration (void *cls,
+ enum GNUNET_BLOCK_Type type,
+ PluginIterator iter,
+ void *iter_cls)
+{
+ struct Plugin *plugin = cls;
+
+ postgres_iterate (plugin, type, GNUNET_YES, 2,
+ iter, iter_cls);
+}
+
+
+
+/**
+ * Select a subset of the items in the datastore and call
+ * the given iterator for each of them.
+ *
+ * @param cls our "struct Plugin*"
+ * @param type entries of which type should be considered?
+ * Use 0 for any type.
+ * @param iter function to call on each matching value;
+ * will be called once with a NULL value at the end
+ * @param iter_cls closure for iter
+ */
+static void
+postgres_plugin_iter_migration_order (void *cls,
+ enum GNUNET_BLOCK_Type type,
+ PluginIterator iter,
+ void *iter_cls)
+{
+ struct Plugin *plugin = cls;
+
+ postgres_iterate (plugin, 0, GNUNET_NO, 3,
+ iter, iter_cls);
+}
+
+
+
+/**
+ * Select a subset of the items in the datastore and call
+ * the given iterator for each of them.
+ *
+ * @param cls our "struct Plugin*"
+ * @param type entries of which type should be considered?
+ * Use 0 for any type.
+ * @param iter function to call on each matching value;
+ * will be called once with a NULL value at the end
+ * @param iter_cls closure for iter
+ */
+static void
+postgres_plugin_iter_all_now (void *cls,
+ enum GNUNET_BLOCK_Type type,
+ PluginIterator iter,
+ void *iter_cls)
+{
+ struct Plugin *plugin = cls;
+
+ postgres_iterate (plugin,
+ 0, GNUNET_YES, 0,
+ iter, iter_cls);
+}
+
+
+/**
+ * Drop database.
+ */
+static void
+postgres_plugin_drop (void *cls)
+{
+ struct Plugin *plugin = cls;
+
+ pq_exec (plugin, "DROP TABLE gn080", __LINE__);
+}
+
+
+/**
+ * Entry point for the plugin.
+ *
+ * @param cls the "struct GNUNET_DATASTORE_PluginEnvironment*"
+ * @return our "struct Plugin*"
+ */
+void *
+libgnunet_plugin_datastore_postgres_init (void *cls)
+{
+ struct GNUNET_DATASTORE_PluginEnvironment *env = cls;
+ struct GNUNET_DATASTORE_PluginFunctions *api;
+ struct Plugin *plugin;
+
+ plugin = GNUNET_malloc (sizeof (struct Plugin));
+ plugin->env = env;
+ if (GNUNET_OK != init_connection (plugin))
+ {
+ GNUNET_free (plugin);
+ return NULL;
+ }
+ api = GNUNET_malloc (sizeof (struct GNUNET_DATASTORE_PluginFunctions));
+ api->cls = plugin;
+ api->get_size = &postgres_plugin_get_size;
+ api->put = &postgres_plugin_put;
+ api->next_request = &postgres_plugin_next_request;
+ api->get = &postgres_plugin_get;
+ api->update = &postgres_plugin_update;
+ api->iter_low_priority = &postgres_plugin_iter_low_priority;
+ api->iter_zero_anonymity = &postgres_plugin_iter_zero_anonymity;
+ api->iter_ascending_expiration = &postgres_plugin_iter_ascending_expiration;
+ api->iter_migration_order = &postgres_plugin_iter_migration_order;
+ api->iter_all_now = &postgres_plugin_iter_all_now;
+ api->drop = &postgres_plugin_drop;
+ GNUNET_log_from (GNUNET_ERROR_TYPE_INFO,
+ "postgres", _("Postgres database running\n"));
+ return api;
+}
+
+
+/**
+ * Exit point from the plugin.
+ * @param cls our "struct Plugin*"
+ * @return always NULL
+ */
+void *
+libgnunet_plugin_datastore_postgres_done (void *cls)
+{
+ struct GNUNET_DATASTORE_PluginFunctions *api = cls;
+ struct Plugin *plugin = api->cls;
+
+ if (plugin->next_task != GNUNET_SCHEDULER_NO_TASK)
+ {
+ GNUNET_SCHEDULER_cancel (plugin->env->sched,
+ plugin->next_task);
+ plugin->next_task = GNUNET_SCHEDULER_NO_TASK;
+ GNUNET_free (plugin->next_task_nc);
+ plugin->next_task_nc = NULL;
+ }
+ PQfinish (plugin->dbh);
+ GNUNET_free (plugin);
+ GNUNET_free (api);
+ return NULL;
+}
+
+/* end of plugin_datastore_postgres.c */
Added: gnunet/src/datastore/test_datastore_api_data_postgres.conf
===================================================================
--- gnunet/src/datastore/test_datastore_api_data_postgres.conf
(rev 0)
+++ gnunet/src/datastore/test_datastore_api_data_postgres.conf 2010-08-25
19:10:18 UTC (rev 12734)
@@ -0,0 +1,55 @@
+[PATHS]
+SERVICEHOME = /tmp/test-gnunet-datastore-postgres/
+
+[arm]
+DEFAULTSERVICES =
+PORT = 42466
+HOSTNAME = localhost
+
+[resolver]
+PORT = 42464
+HOSTNAME = localhost
+
+[datastore]
+PORT = 22654
+HOSTNAME = localhost
+HOME = $SERVICEHOME
+CONFIG = $DEFAULTCONFIG
+BINARY = gnunet-service-datastore
+ACCEPT_FROM = 127.0.0.1;
+ACCEPT_FROM6 = ::1;
+QUOTA = 10000000
+BLOOMFILTER = $SERVICEHOME/fs/bloomfilter
+DATABASE = postgres
+DISABLE_SOCKET_FORWARDING = YES
+# USERNAME =
+# MAXBUF =
+# TIMEOUT =
+# DISABLEV6 =
+# BINDTO =
+# REJECT_FROM =
+# REJECT_FROM6 =
+# PREFIX =
+# DEBUG = YES
+# PREFIX = xterm -T datastore -e gdb --args
+PREFIX = valgrind --tool=memcheck --leak-check=yes
+BINARY = gnunet-service-datastore
+
+[datastore-postgres]
+CONFIG = dbname=gnunetcheck
+# USER =
+# PASSWORD =
+# HOST =
+# PORT =
+
+[statistics]
+PORT = 22667
+HOSTNAME = localhost
+
+[testing]
+WEAKRANDOM = YES
+
+[dht]
+AUTOSTART = NO
+
+
Property changes on: gnunet/src/vpn
___________________________________________________________________
Modified: svn:ignore
- gnunet-helper-vpn
gnunet-helper-hijack-dns
Makefile
Makefile.in
gnunet-vpn-helper
gnunet-daemon-vpn
.deps
+ gnunet-service-dns
gnunet-helper-vpn
gnunet-helper-hijack-dns
Makefile
Makefile.in
gnunet-vpn-helper
gnunet-daemon-vpn
.deps
| [Prev in Thread] |
Current Thread |
[Next in Thread] |
- [GNUnet-SVN] r12734 - in gnunet: . contrib doc src/datastore src/vpn,
gnunet <=