Re: Someone interested in writing a regression test suite for Hurd components?

[Thomas Schwinge]

Hello!

On Wed, Sep 23, 2009 at 11:15:23PM +0530, Shakthi Kannan wrote:
> --- On Wed, Jul 22, 2009 at 4:46 PM, Thomas Schwinge <tschwinge@gnu.org> 
> wrote:
> | Before installing these patches, I'd like to have a basic regression test
> | suite set-up that gives us some confidence that they don't in turn cause
> | breakage on other parts / uses of rpctrace.
> 
> Could you furnish more details?

<http://en.wikipedia.org/wiki/Regression_testing>

> | Now, testing rpctrace is not like testing libpipe, where you simply come
> | up with some testing programs that use the libpipe C API and do some
> | consistency checks alongside.  Testing rpctrace means running it against
> | testee programs, and then parse rpctrace's output and compare that
> | against a to-be-expected version.  Some fuzzy comparisions will be
> | needed, as PIDs and all sort of other things will vary between two runs
> | of the same testee.  Also some machinery to take care of hanging rpctrace
> | processes will be needed.
> 
> Could you give one documented example?

Running rpctrace on /bin/true, ten lines somewhere in the middle:

     120->thread_resume () = 0 
    task8175->task_set_special_port (3  110) = 0 
     118->proc_setmsgport_request ( 110) = 0  (null)
     118->proc_set_arg_locations_request (16928260 16928268) = 0 
    task8175->mach_port_allocate (1) = 0 pn{ 16}
    task8175->mach_port_allocate (1) = 0 pn{ 17}
    task8175->mach_port_insert_right (pn{ 16}               [0] = pass through 
port 123, type 17
    ) = 0 
    task8175->mach_port_set_qlimit (pn{ 17} 1) = 0 
     118->proc_handle_exceptions_request ( 125  126 5 {75 31 31 31 0 0 0 0 0 0 
0 0 17206368 23 0 18611680 0}) = 0 

Running it on another system:

     120->thread_resume () = 0 
    task7005->task_set_special_port (3  110) = 0 
     118->proc_setmsgport_request ( 110) = 0  (null)
     118->proc_set_arg_locations_request (16928212 16928220) = 0 
    task7005->mach_port_allocate (1) = 0 pn{ 16}
    task7005->mach_port_allocate (1) = 0 pn{ 17}
    task7005->mach_port_insert_right (pn{ 16}               [0] = pass through 
port 123, type 17
    ) = 0 
    task7005->mach_port_set_qlimit (pn{ 17} 1) = 0 
     118->proc_handle_exceptions_request ( 125  126 5 {75 31 31 31 0 0 0 0 0 0 
0 0 17205344 23 0 18587008 0}) = 0 

These two runs should evaluate to being equal, but are clearly not equal
character by character.


> | I guess that instead of starting to write shell scripts etc. for handling
> | all this, something like expect, <http://expect.nist.gov/>, or DejaGnu,
> | <http://www.gnu.org/software/dejagnu/>, should instead be used for this
> | task, but I have no experience with these.  Is someone interested in
> | working on that?
> \--
> 
> Can we use Check (LGPL)?
> http://check.sourceforge.net/

Yes, from a quick glance that also seems to fit the bell for a certain
sort of tests, but not for the rpctrace example -- or does it?  (I didn't
look too closely.)


Regards,
 Thomas

signature.asc
Description: Digital signature