[GNUnet-SVN] r9988 - in Extractor: . doc src/main

[gnunet]

Author: grothoff
Date: 2010-01-13 15:38:24 +0100 (Wed, 13 Jan 2010)
New Revision: 9988

Removed:
   Extractor/PLATFORMS
   Extractor/README.debian
   Extractor/README.macosx
Modified:
   Extractor/README
   Extractor/doc/extract.1
   Extractor/doc/extractor.texi
   Extractor/src/main/extract.c
Log:
documentation update

Deleted: Extractor/PLATFORMS
===================================================================
--- Extractor/PLATFORMS 2010-01-13 13:52:34 UTC (rev 9987)
+++ Extractor/PLATFORMS 2010-01-13 14:38:24 UTC (rev 9988)
@@ -1,58 +0,0 @@
-This file lists the known problems with libextractor on various
-platforms.  Let us know if you know more about it than we do.
-
-GNU/Linux (x86):
-- perfect, but with gcc 3.3 you must turn off optimizations!
-
-GNU/Linux (32-/64-bit PowerPC):
-- perfect
-
-GNU/Linux (Ultra Sparc):
-- perfect
-
-GNU/Linux (Alpha):
-- perfect
-
-GNU/Linux (x86_64, Fedora Core)
-- set "CFLAGS=-fPIC" before running configure
-
-FreeBSD (x86):
-- perfect
-
-OpenBSD 3.8:
-- Chan Siu Ming <address@hidden> writes:
-  - libtool is good to have
-  - OpenBSD 3.8 also doesn't have CODESET in langinfo.h (used in about three 
places.) I'm not sure how to cleanly overcome this.
-  - for some reason, convert.a is linked with a static version of libiconv.a, 
so that convert.a ends up containing convert.o
-    and libiconv.a, which confuses the linker, because archives aren't 
supposed to contain other archives. Removed the ICONV
-    stuff from convert_la_LDFLAGS in some Makefiles and it seemed to get past 
that.
-
-- NetBSD:
-- unknown
-
-OS X (10.3, PowerPC):
-- LIBEXTRACTOR_PREFIX needs to be set properly for 'make check'
-- (X)PDF extractor does not work properly, especially with gcc 3.x
-  (use the C-based pdfextractor (default now anyway))
-
-Solaris 8 (SunOS 5.8, Sparc SunFire 880):
-- unknown
-
-Solaris 10
-- reported success for 0.5.11 with minor modifiations (on LE mailinglist, 
2006/3/27)
-
-MinGW:
-- linking against libstdc++ fails
-    my workaround: rename libstdc++.la
-    Gerrit P. Haase writes:
-      Linking -lstdc++ with the provided libtool fails also on Cygwin, this
-      is a problem with libtool, there is unfortunately no flag to tell
-      libtool how to do its job on Cygwin and it seems that it cannot be the
-      default to set the library check to 'pass_all'.  Workaround as you
-      suggested doesn't work on Cygwin, IIRC.  I use a patched libtool for
-      these kind of problems.
-
-Cygwin:
-- compiles fine after touch src/plugins/printable/config.h
-- plugins do not load (weird Win32 error code)
-

Modified: Extractor/README
===================================================================
--- Extractor/README    2010-01-13 13:52:34 UTC (rev 9987)
+++ Extractor/README    2010-01-13 14:38:24 UTC (rev 9988)
@@ -10,6 +10,7 @@
 
 libextractor is a GNU package (http://www.gnu.org/).
 
+More detailed documentation is available in the GNU libextractor manual.
 
 
 extract
@@ -31,57 +32,3 @@
 When building libextractor binaries, please make sure all of these
 dependencies are available.  Otherwise the build system may
 automatically build only a subset of libextractor.
-
-
-
-Writing plugins
-===============
-
-
-If you want to write your own extractor for some filetype, all you
-need to do is write a little library that implements a single method
-with this signature:
-
-
-int
-EXTRACTOR_XXX_extract
-   (const char *data,
-    size_t data_size,
-    EXTRACTOR_MetaDataProcessor proc,
-    void *proc_cls,
-    const char * options);
-
-where XXX is the name of the library file that you will tell
-libextractor to load, minus the suffix.  For example, if you link your
-extractor into a file called 'libextractor_my.so', the method above should
-be called 'EXTRACTOR_my_extract'.
-
-data is a pointer to the contents of the file and data_size is the
-size of data.  The extract method must call the proc function with all
-of the meta data found.  
-
-An example implementation can be found in mp3_extractor.c.
-
-
-
-Notes
-=====
-
-On Mac OS X, libextractor will avoid using GCC 3.1, because of
-problems compiling one of the extractors.  GCC 3.3 and 2.95.2 are
-known to work well; as such, libextractor will first look for 3.3 (by
-attempting to run gcc-3.3, cpp-3.3, and g++-3.3) and then 2.95.2 (by
-attempting to run gcc2 and g++2).
-
-If libextractor fails to find the plugins, a possible method of last
-resort is to set the environment variable LIBEXTRACTOR_PREFIX to the
-parent of the directory where the plugins are installed (i.e., if the
-plugins are in "/foo/bar/lib/libextractor/*.so", set the variable to
-"/foo/bar/lib").  This should not be needed if "extract" is in
-"/foo/bar/bin/extract" and "/foo/bar/bin" is in the PATH, if you are
-running Linux and "libextractor.so" is in
-"/foo/bar/lib/libextractor.so", or if you are on linux and the binary
-using libextractor resides in "/foo/bar/bin", or if you are under
-Windows and "GetModuleFileName" returns "/foo/bar/bin".  If none of
-these common circumstances apply, you may have to set the environment
-variable.

Deleted: Extractor/README.debian
===================================================================
--- Extractor/README.debian     2010-01-13 13:52:34 UTC (rev 9987)
+++ Extractor/README.debian     2010-01-13 14:38:24 UTC (rev 9988)
@@ -1,10 +0,0 @@
-This is a list of debian (unstable) packages that you may want to
-install prior to compiling GNU libextractor.
-
-=================================================
-
-Absolutely needed:
-
-
-==============================================
-

Deleted: Extractor/README.macosx
===================================================================
--- Extractor/README.macosx     2010-01-13 13:52:34 UTC (rev 9987)
+++ Extractor/README.macosx     2010-01-13 14:38:24 UTC (rev 9988)
@@ -1,65 +0,0 @@
-libextractor on Mac OS X
-========================
-
-libextractor has two installation methods on Mac OS X: it can be installed
-as a Mac OS X framework or with the standard `./configure; make; make install'
-shell commands. The framework package is self-contained, but currently omits
-some of the extractor plugins that can be compiled in if libextractor is
-installed with ./configure; make; make install' (provided that the required
-dependencies exist.)
-
-Installing and uninstalling the framework
-=========================================
-
-The binary framework is distributed as a disk image (Extractor-x.x.xx.dmg).
-Installation is done by opening the disk image and clicking Extractor.pkg
-inside it. The Mac OS X installer application will then run. The framework
-is installed to the root volume's /Library/Frameworks folder and installing
-will require admin privileges.
-
-The framework can be uninstalled by dragging
-/Library/Frameworks/Extractor.framework to Trash.
-
-Using the framework
-===================
-
-In the framework, the 'extract' command line tool can be found at 
-/Library/Frameworks/Extractor.framework/Versions/Current/bin/extract
-
-The framework can be used in software projects as a framework or as a dynamic
-library. 
-Framework usage:
->>> hello.c
-#include <Extractor/extractor.h>
-
-int main()
-{
-       EXTRACTOR_ExtractorList *el;
-       el = EXTRACTOR_loadDefaultLibraries ();
-       EXTRACTOR_removeAll (el);
-       return 0;
-}
-gcc -o hello hello.c -framework Extractor
-
-Dynamic library usage:
->>> hello.c
-#include <extractor.h>
-
-int main()
-{
-       EXTRACTOR_ExtractorList *el;
-       el = EXTRACTOR_loadDefaultLibraries ();
-       EXTRACTOR_removeAll (el);
-       return 0;
-}
-gcc -I/Library/Frameworks/Extractor.framework/Versions/Current/include -o 
hello hello.c -L/Library/Frameworks/Extractor.framework/Versions/Current/lib 
-lextractor
-
-Notice the difference in the #include line.
-
-When using the framework as a dynamic library in projects using autotools,
-one would most likely want to add 
-"-I/Library/Frameworks/Extractor.framework/Versions/Current/include"
-to CPPFLAGS and 
-"-L/Library/Frameworks/Extractor.framework/Versions/Current/lib"
-to LDFLAGS.
-

Modified: Extractor/doc/extract.1
===================================================================
--- Extractor/doc/extract.1     2010-01-13 13:52:34 UTC (rev 9987)
+++ Extractor/doc/extract.1     2010-01-13 14:38:24 UTC (rev 9988)
@@ -46,9 +46,7 @@
 .SH OPTIONS
 .TP 8
 .B \-b
-Display the output in BiBTeX format. This implies the
-.B \-d
-option
+Display the output in BiBTeX format.
 .TP 8
 .B \-g
 Use grep\-friendly output (all keywords on a single line for each file).  Use 
the verbose option to print the filename first, followed by the keywords.  Use 
the verbose option twice to also display the keyword types.  This option will 
not print keyword types or non\-textual metadata.
@@ -56,11 +54,8 @@
 .B \-h
 Print a brief summary of the options.
 .TP 8
-.B \-H ALGORITHM
-Use the ALGORITHM to compute a hash of each file (possible algorithms are sha1 
and md5).
-.TP 8
 .B \-i
-Run plugins in-process (for debugging).  By default, each plugin is run in its 
own process.
+Run plugins in\-process (for debugging).  By default, each plugin is run in 
its own process.
 .TP 8
 .BI \-l " libraries"
 Use the specified libraries to extract keywords. The general format of 
libraries is .I [[\-]LIBRARYNAME[:[\-]LIBRARYNAME]*] where LIBRARYNAME is a 
libextractor compatible library and typically of the form .Ijpeg\. The minus 
before the libraryname indicates that this library should be removed from the 
existing list.  To run only a few selected plugins, use \-l in combination with 
\-n.
@@ -99,7 +94,7 @@
 $ extract \-p comment test/test.jpg
 comment \- (C) 2001 by Christian Grothoff, using gimp 1.2 1
 
-$ extract \-nV \-l libextractor_png.so \-p comment test/test.jpg test/test.png
+$ extract \-nV \-l png.so \-p comment test/test.jpg test/test.png
 Keywords for file test/test.jpg:
 Keywords for file test/test.png:
 comment \- Testing keyword extraction

Modified: Extractor/doc/extractor.texi
===================================================================
--- Extractor/doc/extractor.texi        2010-01-13 13:52:34 UTC (rev 9987)
+++ Extractor/doc/extractor.texi        2010-01-13 14:38:24 UTC (rev 9988)
@@ -171,11 +171,16 @@
 @node Preparation
 @chapter Preparation
 
-Compiling @gnule{} follows the standard GNU autotools
-build process using @command{configure} and @command{make}.  For
-details, read the @file{INSTALL} file and query 
address@hidden|./configure --help|} for additional options.
+This chapter first describes the general build instructions that
+should apply to all systems.  Specific instructions for known problems
+for particular platforms are then described in individual sections
+afterwards.
 
+Compiling @gnule{} follows the standard GNU autotools build process
+using @command{configure} and @command{make}.  For details on the GNU
+autotools build process, read the @file{INSTALL} file and query
address@hidden|./configure --help|} for additional options.  
+
 @gnule{} has various dependencies, some of which are optional. 
 Instead of specifying the names of the software packages, we
 will give the list in terms of the names of the respective
@@ -271,6 +276,135 @@
 @gnule{} cannot locate a plugin, it will look in
 @verb{|LIBEXTRACTOR_PREFIX/lib/libextractor/|}.
 
+
address@hidden Installation on GNU/Linux
+
+Should work using the standard instructions without problems.
+
+
address@hidden Installation on FreeBSD
+
+Should work using the standard instructions without problems.
+
+
address@hidden Installation on OpenBSD
+
+OpenBSD 3.8 also doesn't have CODESET in @file{langinfo.h}.  CODESET
+is used in @gnule{} in about three places.  This causes problems
+during compilation.
+
+
address@hidden Installation on NetBSD
+
+No reports so far.
+
+
address@hidden Installation using MinGW
+
+Linking -lstdc++ with the provided libtool fails on Cygwin, this
+is a problem with libtool, there is unfortunately no flag to tell
+libtool how to do its job on Cygwin and it seems that it cannot be the
+default to set the library check to 'pass_all'.  Patching libtool may
+help.
+
+Note: this is a rather dated report and may no longer apply.
+
+
address@hidden Installation on OS X
+
+libextractor has two installation methods on Mac OS X: it can be
+installed as a Mac OS X framework or with the standard
address@hidden/configure; make; make install} shell commands. The
+framework package is self-contained, but currently omits some of the
+extractor plugins that can be compiled in if libextractor is installed
+with @command{./configure; make; make install} (provided that the
+required dependencies exist.)
+
address@hidden Installing and uninstalling the framework
+
+The binary framework is distributed as a disk image 
(@file{Extractor-x.x.xx.dmg}).
+Installation is done by opening the disk image and clicking 
@file{Extractor.pkg}
+inside it. The Mac OS X installer application will then run. The framework
+is installed to the root volume's @file{/Library/Frameworks} folder and 
installing
+will require admin privileges.
+
+The framework can be uninstalled by dragging
address@hidden/Library/Frameworks/Extractor.framework} cto the @file{Trash}.
+
+
address@hidden Using the framework
+
+In the framework, the @command{extract} command line tool can be found at 
address@hidden/Library/Frameworks/Extractor.framework/Versions/Current/bin/extract}
+
+The framework can be used in software projects as a framework or as a dynamic
+library. 
+
+When using the framework as a dynamic library in projects using autotools,
+one would most likely want to add 
+"-I/Library/Frameworks/Extractor.framework/Versions/Current/include"
+to CPPFLAGS and 
+"-L/Library/Frameworks/Extractor.framework/Versions/Current/lib"
+to LDFLAGS.
+
+
address@hidden Example for using the framework
+
address@hidden
address@hidden
+// hello.c
+#include <Extractor/extractor.h>
+
+int main()
+{
+  struct EXTRACTOR_PluginList *el;
+  el = EXTRACTOR_plugin_load_defaults (EXTRACTOR_OPTION_DEFAULT_POLICY);
+  // ...
+  EXTRACTOR_plugin_remove_all (el);
+  return 0;
+}
address@hidden verbatim
address@hidden example
+
+You can then compile the example using
+
address@hidden
+$ gcc -o hello hello.c -framework Extractor
address@hidden verbatim
+
address@hidden Example for using the dynamic library
+
address@hidden
address@hidden
+// hello.c
+#include <extractor.h>
+int main()
+{
+  struct EXTRACTOR_PluginList *el;
+  el = EXTRACTOR_plugin_load_defaults (EXTRACTOR_OPTION_DEFAULT_POLICY);
+  // ...
+  EXTRACTOR_plugin_remove_all (el);
+  return 0;
+}
address@hidden verbatim
address@hidden example
+
+You can then compile the example using
+
address@hidden
+$ gcc -I/Library/Frameworks/Extractor.framework/Versions/Current/include \
+  -o hello hello.c \
+  -L/Library/Frameworks/Extractor.framework/Versions/Current/lib \
+  -lextractor
address@hidden verbatim
+
+Notice the difference in the @code{#include} line.
+
+
+
+
+
+
 @section Note to package maintainers
 
 The suggested way to package GNU libextractor is to split it into
@@ -304,6 +438,52 @@
 @node Generalities
 @chapter Generalities
 
address@hidden Introduction to the ``extract'' command
+
+The @command{extract} command takes a list of file names as arguments,
+extracts meta data from each of those files and prints the result to
+the console.  By default, @command{extract} will use all available
+plugins and print all (non-binary) meta data that is found.
+
+The set of plugins used by @command{extract} can be controlled using
+the ``-l'' and ``-n'' options.  Use ``-n'' to not load all of the
+default plugins.  Use ``-l NAME'' to specifically load a certain
+plugin.  For example, specify ``-n -l mime'' to only use the MIME
+plugin.
+
+Using the ``-p'' option the output of @command{extract} can be limited
+to only certain keyword types.  Similarly, using the ``-x'' option,
+certain keyword types can be excluded.  A list of all known keyword
+types can be obtained using the ``-L'' option.
+
+The output format of @command{extract} can be influenced with the
+``-V'' (more verbose, lists filenames), ``-g'' (grep-friendly, all
+meta data on a single line per file) and ``-b'' (bibTeX style)
+options.
+
address@hidden Common usage examples for ``extract''
+
address@hidden
+$ extract test/test.jpg
+comment - (C) 2001 by Christian Grothoff, using gimp 1.2 1
+mimetype - image/jpeg
+
+$ extract -V -x comment test/test.jpg
+Keywords for file test/test.jpg:
+mimetype - image/jpeg
+
+$ extract -p comment test/test.jpg
+comment - (C) 2001 by Christian Grothoff, using gimp 1.2 1
+
+$ extract -nV -l png.so -p comment test/test.jpg test/test.png
+Keywords for file test/test.jpg:
+Keywords for file test/test.png:
+comment - Testing keyword extraction
address@hidden example
+
+
address@hidden Introduction to the libextractor library
+
 Each public symbol exported by @gnule{} has the prefix
 @verb{|EXTRACTOR_|}.  All-caps names are used for constants.  For the
 impatient, the minimal C code for using @gnule{} (on the
@@ -322,6 +502,11 @@
 }
 @end verbatim
 
+The minimal API illustrated by this example is actually sufficient for
+many applications.  The full external C API of @gnule{} is described
+in chapter @xref{Extracting meta data}.  Bindings for other languages
+are described in chapter @xref{Language bindings}.  The API for
+writing new plugins is described in chapter @xref{Writing new Plugins}.
 
 @node Extracting meta data
 @chapter Extracting meta data
@@ -508,7 +693,6 @@
 
 @node Language bindings
 @chapter Language bindings
-
 @cindex Java
 @cindex Mono
 @cindex Perl

Modified: Extractor/src/main/extract.c
===================================================================
--- Extractor/src/main/extract.c        2010-01-13 13:52:34 UTC (rev 9987)
+++ Extractor/src/main/extract.c        2010-01-13 14:38:24 UTC (rev 9988)
@@ -176,8 +176,6 @@
       gettext_noop("produce grep-friendly output (all results on one line per 
file)") },
     { 'h', "help", NULL,
       gettext_noop("print this help") },
-    { 'H', "hash", "ALGORITHM",
-      gettext_noop("compute hash using the given ALGORITHM (currently sha1 or 
md5)") },
     { 'i', "in-process", NULL,
       gettext_noop("run plugins in-process (simplifies debugging)") },
     { 'l', "library", "LIBRARY",
@@ -552,7 +550,6 @@
   int option_index;
   int c;
   char * libraries = NULL;
-  char * hash = NULL;
   int nodefault = NO;
   int defaultAll = YES;
   int bibtex = NO;
@@ -578,7 +575,6 @@
        {"bibtex", 0, 0, 'b'},
        {"grep-friendly", 0, 0, 'g'},
        {"help", 0, 0, 'h'},
-       {"hash", 1, 0, 'H'},
        {"in-process", 0, 0, 'i'},
        {"list", 0, 0, 'L'},
        {"library", 1, 0, 'l'},
@@ -592,7 +588,7 @@
       option_index = 0;
       c = getopt_long (argc,
                       argv, 
-                      "abghH:il:Lnp:vVx:",
+                      "abghil:Lnp:vVx:",
                       long_options,
                       &option_index);
 
@@ -623,9 +619,6 @@
        case 'h':
          printHelp();
          return 0;
-       case 'H':
-         hash = optarg;
-         break;
        case 'i':
          in_process = 1;
          break;
@@ -736,20 +729,6 @@
                                           in_process
                                           ? EXTRACTOR_OPTION_IN_PROCESS
                                           : EXTRACTOR_OPTION_DEFAULT_POLICY);
-  if (hash != NULL) 
-    {
-      name = malloc(strlen(hash) + strlen("hash_") + 1);
-      strcpy(name, "libextractor_hash_");
-      strcat(name, hash);
-      plugins = EXTRACTOR_plugin_add(plugins,
-                                    name,
-                                    NULL,
-                                    in_process
-                                    ? EXTRACTOR_OPTION_IN_PROCESS
-                                    : EXTRACTOR_OPTION_DEFAULT_POLICY);
-      free(name);
-    }
-
   if (processor == NULL)
     processor = &print_selected_keywords;