Re: [Monotone-devel] basic_io inventory

[Stephen Leake]

Stephen Leake <address@hidden> writes:

>>> I can help with the tests and documentation.

I've taken a first stab at updating
tests/automate_inventory/__driver__.lua, and adding some comments in
automate.cc.

As an aside, I found the lua test framework quite nice, once I figured
out how to run it on an individual test.

First the header comment for 'inventory' in automate.cc:

// Name: inventory
// Arguments: [PATH]...
// Added in: 1.0
// Modified to basic_io in: 4.1

// Purpose: Prints a summary of every file or directory found in the
//   workspace or its associated base manifest.
//
// Output: basic_io format. For each item, three to six lines are output:
//
//        path <path>
//    old_node "<old_node_id>" "<old_node path::status>"
//    new_node "<new_node_id>" "<new_node path::status>"
//     fs_type "<current path::status>"
//      status "<status>"
//     changes "{content|attrs}..."
//
// 'fs_type' gives the state of the item in the workspace filesystem.
//
// 'old_node' and 'new_node' give the status of the item in the manifest.
// The 'old_node' and 'new_node' lines are not output if the corresponding
// node state is 'nonexistent'. FIXME: better to output 'none'?
//
// '<status>' is one of 'known', 'missing', 'unknown'
//
// Error conditions: If no workspace book keeping _MTN directory is found,
//   prints an error message to stderr, and exits with status 1.

Is this correct so far?

Here's the first set of checks in
tests/automate_inventory/__driver__.lua. I left the original output
for comparison. I added FIXME: comments where I think the output could
be improved.

My goal here is to get something that is useful for Emacs DVC. It may
be that is different enough from the intent of 'inventory' that there
should be another command for it.

--  not in previous output
--  FIXME: null path is root directory; should eliminate this
check(grep('^    path ""$', "stdout"), 0, false, false)
check(grep('^old_node "1" "directory"$', "stdout"), 0, false, false)
check(grep('^new_node "1" "directory"$', "stdout"), 0, false, false)
check(grep('^ fs_type "none"$', "stdout"), 0, false, false)
check(grep('^  status "missing"$', "stdout"), 0, false, false)
check(grep('^$', "stdout"), 0, false, false)

--  AP 0 0 added
--  FIXME: status should indicate 'added, needs commit'
check(grep('^    path "added"$', "stdout"), 0, false, false)
check(grep('^new_node "2147483648" "file"$', "stdout"), 0, false, false)
check(grep('^ fs_type "file"$', "stdout"), 0, false, false)
check(grep('^  status "known"$', "stdout"), 0, false, false)
check(grep('^$', "stdout"), 0, false, false)

-- D   0 0 dropped
--  FIXME: no 'status'; should indicate 'dropped, needs commit'
check(grep('^    path "dropped"$', "stdout"), 0, false, false)
check(grep('^old_node "2" "file"$', "stdout"), 0, false, false)
check(grep('^ fs_type "none"$', "stdout"), 0, false, false)
check(grep('^$', "stdout"), 0, false, false)

--   I 0 0 ignored
--  FIXME: need an option to not output these
check(grep('^   path "ignored~"$', "stdout"), 0, false, false)
check(grep('^fs_type "file"$', "stdout"), 0, false, false)
check(grep('^ status "ignored"$', "stdout"), 0, false, false)
check(grep('^$', "stdout"), 0, false, false)

-- not in original output (why not?)
check(grep('^   path "inventory_hooks.lua"$', "stdout"), 0, false, false)
check(grep('^fs_type "file"$', "stdout"), 0, false, false)
check(grep('^ status "unknown"$', "stdout"), 0, false, false)
check(grep('^$', "stdout"), 0, false, false)

-- not in original output (why not?)
check(grep('^   path "keys"$', "stdout"), 0, false, false)
check(grep('^fs_type "directory"$', "stdout"), 0, false, false)
check(grep('^ status "unknown"$', "stdout"), 0, false, false)
check(grep('^$', "stdout"), 0, false, false)

-- not in original output (why not?)
check(grep('^   path "keys/address@hidden"$', "stdout"), 0, false, false)
check(grep('^fs_type "file"$', "stdout"), 0, false, false)
check(grep('^ status "unknown"$', "stdout"), 0, false, false)
check(grep('^$', "stdout"), 0, false, false)

-- not in original output (why not?)
check(grep('^   path "min_hooks.lua"$', "stdout"), 0, false, false)
check(grep('^fs_type "file"$', "stdout"), 0, false, false)
check(grep('^ status "unknown"$', "stdout"), 0, false, false)
check(grep('^$', "stdout"), 0, false, false)

--   M 0 0 missing
check(grep('^    path "missing"$', "stdout"), 0, false, false)
check(grep('^old_node "3" "file"$', "stdout"), 0, false, false)
check(grep('^new_node "3" "file"$', "stdout"), 0, false, false)
check(grep('^ fs_type "none"$', "stdout"), 0, false, false)
check(grep('^  status "missing"$', "stdout"), 0, false, false)
check(grep('^$', "stdout"), 0, false, false)

-- R   1 0 original
-- FIXME: no 'status'; should indicate 'renamed, needs commit'
check(grep('^    path "original"$', "stdout"), 0, false, false)
check(grep('^old_node "4" "file"$', "stdout"), 0, false, false)
check(grep('^ fs_type "none"$', "stdout"), 0, false, false)
check(grep('^$', "stdout"), 0, false, false)

--   P 0 0 patched
check(grep('^    path "patched"$', "stdout"), 0, false, false)
check(grep('^old_node "5" "file"$', "stdout"), 0, false, false)
check(grep('^new_node "5" "file"$', "stdout"), 0, false, false)
check(grep('^ fs_type "file"$', "stdout"), 0, false, false)
check(grep('^  status "known"$', "stdout"), 0, false, false)
check(grep('^ changes "content"$', "stdout"), 0, false, false)
check(grep('^$', "stdout"), 0, false, false)

--  R  0 1 renamed
-- FIXME: status should indicate 'renamed, needs commit'
check(grep('^    path "renamed"$', "stdout"), 0, false, false)
check(grep('^new_node "4" "file"$', "stdout"), 0, false, false)
check(grep('^ fs_type "file"$', "stdout"), 0, false, false)
check(grep('^  status "known"$', "stdout"), 0, false, false)
check(grep('^$', "stdout"), 0, false, false)

-- not in original output (why not?)
check(grep('^   path "test.db"$', "stdout"), 0, false, false)
check(grep('^fs_type "file"$', "stdout"), 0, false, false)
check(grep('^ status "unknown"$', "stdout"), 0, false, false)
check(grep('^$', "stdout"), 0, false, false)

-- not in original output (why not?)
check(grep('^   path "test_hooks.lua"$', "stdout"), 0, false, false)
check(grep('^fs_type "file"$', "stdout"), 0, false, false)
check(grep('^ status "unknown"$', "stdout"), 0, false, false)
check(grep('^$', "stdout"), 0, false, false)

-- not in original output (why not?)
check(grep('^   path "tester.log"$', "stdout"), 0, false, false)
check(grep('^fs_type "file"$', "stdout"), 0, false, false)
check(grep('^ status "unknown"$', "stdout"), 0, false, false)
check(grep('^$', "stdout"), 0, false, false)

-- not in original output (why not?)
check(grep('^   path "ts-stderr"$', "stdout"), 0, false, false)
check(grep('^fs_type "file"$', "stdout"), 0, false, false)
check(grep('^ status "unknown"$', "stdout"), 0, false, false)
check(grep('^$', "stdout"), 0, false, false)

-- not in original output (why not?)
check(grep('^   path "ts-stdin"$', "stdout"), 0, false, false)
check(grep('^fs_type "file"$', "stdout"), 0, false, false)
check(grep('^ status "unknown"$', "stdout"), 0, false, false)
check(grep('^$', "stdout"), 0, false, false)

-- not in original output (why not?)
check(grep('^   path "ts-stdout"$', "stdout"), 0, false, false)
check(grep('^fs_type "file"$', "stdout"), 0, false, false)
check(grep('^ status "unknown"$', "stdout"), 0, false, false)
check(grep('^$', "stdout"), 0, false, false)

--     0 0 unchanged
-- FIXME: need option to not output this
check(grep('^    path "unchanged"$', "stdout"), 0, false, false)
check(grep('^old_node "6" "file"$', "stdout"), 0, false, false)
check(grep('^new_node "6" "file"$', "stdout"), 0, false, false)
check(grep('^ fs_type "file"$', "stdout"), 0, false, false)
check(grep('^  status "known"$', "stdout"), 0, false, false)
check(grep('^$', "stdout"), 0, false, false)

--   U 0 0 unknown
check(grep('^   path "unknown"$', "stdout"), 0, false, false)
check(grep('^fs_type "file"$', "stdout"), 0, false, false)
check(grep('^ status "unknown"$', "stdout"), 0, false, false)
check(grep('^$', "stdout"), 0, false, false)


These checks pass; the rest of the test still needs work. The rest of
the test cases explore all the corner cases.

In summary, I'd like to see the following changes in the output:

1) clearly indicate whether the item needs committing or not.

2) clearly indicate renamed and dropped.

3) option to not output unchanged and ignored files (to speed processing)

A couple other points/questions;

The objection to having optional output seems to be that it will
complicate maintaining the code. That is true, but the gain will be
quite significant. Committing files should be fast and easy; getting a
list of the current files that need attention is the first step in
committing (at least, the way I do things). So it needs to be as fast
as possible. 26 seconds is already pretty long.

It would be more consistent to output 'old_node <id> "none"' rather
than not outputing the old_node line; same for new_node. But that's
not hard to handle in the Emacs parser.

I'm not clear what the rationale is for the node ids; what subsequent
commands would use them?

There are several items that appear in this output that are not in the
original output. I think the current output is correct; I don't
understand why they were not in the original output. Can anyone shed
light on this?

--
-- Stephe