Re: [Axiom-developer] Axisp news

[Stephen Wilson]

Cliff,

C Y <address@hidden> writes:

> --- Stephen Wilson <address@hidden> wrote:
> 
> > Any suggestions, preferences, comments, etc very much appreciated!
> 
> It looks like Lift has had a new release within the last few days - it
> might be worthwhile to inquire if the performance issues have been
> ironed out since the comparison article was written?  I agree it sounds
> very interesting but I think we should probably first figure out if the
> performance drop is something that can be avoided.


As you know, I discovered that Lift, despite initial measurements,
does not currently scale very well.

I discovered the principle bottleneck, and implemented a fix.

ASSEMBLE-TESTS is a macro which generates n assorted, simple minded
tests.  Tests are run by default at the point of definition, and can
be re-executed at any time via RUN-TESTS.

*** Here are some timings from the original Lift:

  >(time (assemble-tests 50))

  real time       :      7.520 secs
  run-gbc time    :      2.480 secs
  child run time  :      0.980 secs
  gbc time        :      0.390 secs
  #<Test passed>

  >(time (run-tests))

  Start: LIFT-EXAMPLES-1
  real time       :      1.700 secs
  run-gbc time    :      0.450 secs
  child run time  :      0.000 secs
  gbc time        :      0.150 secs
  #<Results for LIFT-EXAMPLES-1 [50 Successful tests]>
  
  >(time (assemble-tests 100))

  real time       :     32.310 secs
  run-gbc time    :     14.580 secs
  child run time  :      0.000 secs
  gbc time        :      1.360 secs
  #<Test passed>

  >(time (run-tests))

  Start: LIFT-EXAMPLES-1
  real time       :      2.410 secs
  run-gbc time    :      1.000 secs
  child run time  :      0.000 secs
  gbc time        :      0.170 secs
  #<Results for NIL [100 Successful tests]>


*** And some timings with my optimization:

  >(time (assemble-tests 100))

  real time       :      0.840 secs
  run-gbc time    :      0.310 secs
  child run time  :      0.000 secs
  gbc time        :      0.120 secs
  #<Test passed>

  >(time (run-tests))

  Start: LIFT-EXAMPLES-1
  real time       :      0.700 secs
  run-gbc time    :      0.220 secs
  child run time  :      0.000 secs
  gbc time        :      0.130 secs
  #<Results for LIFT-EXAMPLES-1 [100 Successful tests]>
  
  >(time (assemble-tests 2500))
  
  real time       :     33.300 secs
  run-gbc time    :     14.740 secs
  child run time  :      0.000 secs
  gbc time        :      1.230 secs
  #<Test passed>
  
  >(time (run-tests))
  
  Start: LIFT-EXAMPLES-1
  real time       :     14.470 secs
  run-gbc time    :      5.660 secs
  child run time  :      0.000 secs
  gbc time        :      0.850 secs
  #<Results for LIFT-EXAMPLES-1 [2500 Successful tests]>


As you can see, we can now define 2500 tests in approximately the same
amount of time as 100 tests used to take.

There are other optimizations available, but I am not pressed to
implement them.  I am fairly confident that we can use Lift to support
thousands of tests for Axiom.

I will submit a patch to the author of Lift, including this
optimization and the port to GCL.  There are a few remaining wrinkles
to work out but should be able to have something soon-ish.


Take care,
Steve