Script to identify mach dependencies

[Farid Hajji]

Hi,

[I've already mailed this to help-hurd a while ago, but I think that it
can be useful here too.]

The following script is a front-end to global(1) and friends. It creates
a cross-reference listing of every symbol defined in gnumach w.r.t. hurd,
mig, glibc. When calling this script, direct stderr and stdout to different
files.

This script can be useful to identify the mach calls that are used in
the Hurd. It can be considered zeroth-step in writing libmachemu and
l4-mach.

The output (stdout) of this script looks somewhat like this:

----- cut here -------- cut here --------- cut here --------- cut here -----
<...>
device_write:
        ./gnumach/linux/dev/glue/net.c:410:device_write (void *d, ipc_port_t rep
ly_port,
        ./gnumach/linux/dev/glue/block.c:1306:device_write (void *d, ipc_port_t 
reply_port,
        ./gnumach/device/ds_routines.c:420:device_write(d, reply_port, reply_por
t_type, mode, recnum,

        hurd/ext2fs/devio.c     32      if (device_write (device_port, 0, device
_start + addr,
        hurd/libstore/device.c  63      return dev_error (device_write (store->p
ort, 0, addr,
        hurd/pfinet/ethernet.c  188     err = device_write (edev->ether_port, D_
NOWAIT, 0, skb->data, skb->len, &count);
        hurd/serverboot/file_io.c       308     rc = device_write(
<...>
----- cut here -------- cut here --------- cut here --------- cut here -----

After the symbol name comes the line in gnumach tree, where the symbol is
defined. Following a blank line, the lines in hurd, glibc, mig using that
symbol are listed.

How to read the output?

  * use cut(1) to view the output in a window more easily.
  * scroll over the ALL-CAPS symbols. The interesting stuff
    starts with lowercase symbols.
  * The script has no way to identify the external interface of gnumach.
    You'll find a list of interesting symbols to examine at the end of
    the script.


----- cut here -------- cut here --------- cut here --------- cut here -----
#!/usr/bin/perl -w
# mach-crossref.pl -- determines the interface of the Hurd to GNUmach.
# Farid Hajji <address@hidden>
# $Id: crossref.pl,v 1.1 2000/07/29 18:15:10 farid Exp $

# Instructions for using this script:

# -1. get and install global(1) and friends if it's not already
#     present on your system.
#     global is present on FreeBSD by default.
#     global home page: http://wafu.netgate.net/tama/unix/global.html
# 0.  cd to a directory of your choice (let's call it <topdir>)
# 1.  get via cvs the current sources of glibc, hurd, gnumach, mig and grub
#     from subversions.gnu.org like this:
#       cd <topdir>
#       cvs -d :pserver:address@hidden:/cvs checkout grub
#       cvs -d :pserver:address@hidden:/cvs checkout mig
#       cvs -d :pserver:address@hidden:/cvs checkout gnumach
#       cvs -d :pserver:address@hidden:/cvs checkout hurd
#       cvs -d :pserver:address@hidden:/cvs checkout glibc
#     this creates the directories <topdir>/glibc, <topdir>/hurd, ...
# 1'. alternatively update your cvs sources like this:
#       cd <topdir>
#       cd glibc;   cvs update; cd ..
#       cd gnumach; cvs update; cd ..
#       cd hurd;    cvs update; cd ..
#       cd mig;     cvs update; cd ..
#       cd grub;    cvs update; cd ..
# 2.  call gtags to generate the cross reference indexes.
#     important: call gtags in <topdir>!
#       cd <topdir>; gtags
#     note that the generated files <topdir>/{GPATH,GRTAGS,GSYMS,GTAGS}
#     can get quite large:
#       -rw-r--r--   2 farid    users     1777664 Jul 29 16:46 GPATH
#       -rw-r--r--   2 farid    users    25378816 Jul 29 16:55 GRTAGS
#       -rw-r--r--   1 farid    users    32382976 Jul 29 17:17 GSYMS
#       -rw-r--r--   2 farid    users     5529600 Jul 29 16:46 GTAGS
# 3.  [optional step] If you like browsing HTMLized sources, generate
#     <topdir>/HTML subtree with a call to htags:
#       cd <topdir>; htags -fhvat 'GNU/Hurd Project Sources'
#     Here again, the HTML subdirectory can get very large.
#       address@hidden:~/Devel/GNU-HURD> du -s -k HTML
#       181182  HTML
# 4.  now, call this script in <topdir>.
#     debug messages go to stderr, normal output to stdout.
#     You may wish to read the cross-reference in a large xterm window ;-)

# CAVEATS:
# 1. This script relies to 100% upon the tags generated by gtags(1)
#    which may well be uncomplete.
# 2. This script doesn't run mig(1) on the .defs files. Therefore,
#    some calls to Mach functions generated by MiG when translating
#    .defs into .c and .h will be missing. You could try to run MiG
#    to generate .c and .h files _before_ running gtags (Step 2 above).
# 3. weak_alias() not dealt with correctly. The glibc often defines
#    function names starting with (1 or 2) underscores and then defines
#    in weak_alias() directives underscore-less aliases for those
#    functions. Following weak_alias() can be a trivial enhancement
#    to this script, but its not done yet.
# 4. Certain mach symbols don't belong to Mach's external interface.
#    If such symbols ever appear outside of gnumach (say glibc, hurd),
#    they will be referenced!
# 5. Hardcoded is the gnumach subdirectory. This script will break
#    if this name is ever changed or if a gnumach subdirectory is
#    added somewhere outside of gnumach (say in glibc or hurd;
#    fortunately, only 'mach' is used for now!).
# 6. All-caps SYMBOLS are a bit bogus. The most interesting part are
#    lower case function names which appear in the listing after all
#    uppercase symbols.

%SYMBOLS = get_mach_symbols();
foreach my $symbol (sort keys %SYMBOLS) {
    @REFS = get_references($symbol);
    next unless @REFS;   # Skip symbols that where never used.

    print "-" x 70, "\n";

    # first print locations in gnumach, where symbol is defined
    print "$symbol:\n";
    foreach my $def (@{$SYMBOLS{$symbol}}) {
        print "\t$def\n";
    }

    print "\n";
    
    # now print every reference to $symbol outside of gnumach.
    foreach my $usage (@REFS) {
        ($file, $linenr, $rest) = split(/:/, $usage);
        print "\t$file\t$linenr\t$rest\n";
    }
}

# gets a list of mach symbols (functions and #defines).
# returns a hash indexed by mach symbols, where each value is a ':' joined
# list of ($file, $linenr, $declaration_line).
sub get_mach_symbols {
    my %mach_symbols = ();

    print STDERR "Getting mach symbols...\n";

    open (SYMBOLS, "btreeop GTAGS | grep 'gnumach' |")
        or die "can't call btreeop(1) or GTAGS not available: $!\n";

    while (defined ($line = <SYMBOLS>)) {
        chomp $line;
        ($symbol, $linenr, $file, @rest) = split(/\s+/, $line);

        # XXX multiply defined symbols: use only last declaration for now.
        if (exists $mach_symbols{$symbol}) {
            push(@{$mach_symbols{$symbol}},
                 join(':', $file, $linenr, join(' ', @rest)));

        } else {
            $mach_symbols{$symbol} = [ join(':',
                                            $file, $linenr,
                                            join(' ', @rest)) ];
        }

    }

    close (SYMBOLS)
        or die "problems occured while getting mach symbols: $!\n";

    return %mach_symbols;
}

# returns a list of references, where SYMBOL is being used.
# each entry in the list is a ':' joined list
# of ($file, $linenr, $useage_line)
sub get_references {
    my $symbol = shift;
    my @refs = ();

    print STDERR "Getting references for $symbol...\n";

    open (REFS, "global -rx '^$symbol\$' | grep -v gnumach |")
        or die "can't call global for symbol $symbol: $!\n";

    while (defined ($line = <REFS>)) {
        chomp $line;
        ($symbol, $linenr, $file, @rest) = split(/\s+/, $line);

        push(@refs, join(':', $file, $linenr, join(' ', @rest)));
    }


    close (REFS);    # don't check for return code here, because this will
                     # be the return code of grep missing output, not the
                     # return code of global!
    return @refs;
}


# Looking for interesting Mach symbols in the output listing:
#  dev_*, device_*, host_*, mach_*, memory_object_*, mig_*, mutex_*,
#  spin_*, syscall, task_*, thread_*, vm_*
----- cut here -------- cut here --------- cut here --------- cut here -----

Thanks,

-Farid.

-- 
Farid Hajji -- Unix Systems and Network Admin | Phone: +49-2131-67-555
Broicherdorfstr. 83, D-41564 Kaarst, Germany  | address@hidden
- - - - - - - - - - - - - - - - - - - - - - - + - - - - - - - - - - - -
Murphy's Law fails only when you try to demonstrate it, and thus succeeds.