Re: [Axiom-developer] --patch46

[Camm Maguire]

Greetings!  And please excuse me -- this thread has slipped under the
radar here.

"Bill Page" <address@hidden> writes:

> > > The purpose of this macro is to distinguish the stack address
> > > space. Addresses of local variables should return true, and
> > > others false. GCL tests this on starup on a local variable, NIL,
> > > and core_end. One can see what the macro expands to by doing
> > > 'cpp -I../h main.c' in the o directory.  Most likely there was
> > > some problem in detecting CSTACK_ADDRESS in the configure stage.
> > > I recommend looking at the configure output for this.
> > 
> > Ok. Here is where CSTACK_ADDRESS is mentioned in the configure
> > output:
> > 
> > ...
> > checking for randomized sbrk... no
> > checking finding DBEGIN... got 0x8000000
> > checking finding CSTACK_ADDRESS... got 1073738356
> > ...
> > 

Bill, here is the problem -- Redhat 9 is apparently placing the stack
within the first 1Gb of memory, but you are requesting this amount for
your heap.  GCL won't (or tries not to) allow the head to overrun the
stack: 

p/x -1073750720   /* This is my cstack address.  Just built 2.6.7 with */
$1 = 0xbfffdd40   /* 256*1024 maxpages just fine*/
(gdb) p/x 1073738356 /* here is yours */
$2 = 0x3ffff274
(gdb) p/x 256*1024*4096+0x8000000
$3 = 0x48000000   /* here is the end of your requested heap */
(gdb) 


In all of the machines I've looked at, this is about the worst stack
placement I've seen.  Is this 'enterprise' or 'normal'?   I thought
the former had a sophisticated mechanism to push the stack and the
shared memory area as high up in memory as possible.

I know of no way to change this short of getting a different Linux
kernel, where the issue should be addressable.

One thing we could do is relax the power of two constraint on
maxpages if a lesser amount would suffice.


You say there is another work around with
si::allocate-contiguous-pages, which I suspect to be a feather-edge
manipulation issue, but am not sure.  You can check this out by doing
(room) after your job has completed and seeing how close you are to
OOM. In addition to the contiguous page function you've found, here
are some other ways to explicitly allocate memory to various types:

(si::allocate 'cons 400)
(si::allocate-relocatable-pages 10000)
<will explain the allocate sgc functions later if desired>

The typed pages are self-explanatory I'd think.  Contiguous pages are
used for untyped memory which cannot be moved, typically
compiled-code.  relocatable-pages hold untyped memory to which no
pointer refers directly (i.e. outside of a lisp object, where we can
control the reference in the garbage collector), string bodies, bignum
bodies, certain arrays, etc.

Here is a bit of clarification on the room report:

   211/211    42.9%         CONS RATIO LONG-FLOAT COMPLEX STRUCTURE
   
   ^  pages allocated for this type
        ^  maximum pages before triggering GC
              ^  Percent of the allocated pages holding live objects

=============================================================================
>(room)(si::allocate 'cons 400)(room)

   211/211    42.9%         CONS RATIO LONG-FLOAT COMPLEX STRUCTURE
     2/28     40.8%         FIXNUM SHORT-FLOAT CHARACTER RANDOM-STATE READTABLE 
SPICE
    47/49     72.9%         SYMBOL STREAM
     1/2      12.8%         PACKAGE
     1/38     56.8%         ARRAY HASH-TABLE VECTOR BIT-VECTOR PATHNAME 
CCLOSURE CLOSURE
    16/32     85.8%         STRING
     3/27     98.3%         CFUN BIGNUM
     6/115    88.1%         SFUN GFUN VFUN AFUN CFDATA

   347/512                  contiguous (86 blocks)
       13107                hole
       5242    0.0%         relocatable

       287 pages for cells
     18983 total pages
    101769 pages available
     10320 pages in heap but not gc'd + pages needed for gc marking
    131072 maximum pages

>
T

>
   211/400    44.2%         CONS RATIO LONG-FLOAT COMPLEX STRUCTURE
     2/28     47.5%         FIXNUM SHORT-FLOAT CHARACTER RANDOM-STATE READTABLE 
SPICE
    47/49     72.9%         SYMBOL STREAM
     1/2      12.8%         PACKAGE
     1/38     56.8%         ARRAY HASH-TABLE VECTOR BIT-VECTOR PATHNAME 
CCLOSURE CLOSURE
    16/32     85.8%         STRING
     3/27     98.3%         CFUN BIGNUM
     6/115    88.1%         SFUN GFUN VFUN AFUN CFDATA

   347/512                  contiguous (86 blocks)
       13107                hole
       5242    0.0%         relocatable

       287 pages for cells
     18983 total pages
    101769 pages available
     10320 pages in heap but not gc'd + pages needed for gc marking
    131072 maximum pages

>=============================================================================



Take care,


> > In config.log I see
> > 
> > ...
> > configure:3780: checking for sbrk
> > configure:3798: gcc -o conftest      conftest.c  1>&5
> > configure:3814: checking for randomized sbrk
> > configure:3831: gcc -o conftest      conftest.c  1>&5
> > configure:3863: gcc -o conftest      conftest.c  1>&5
> > configure:3984: checking finding DBEGIN
> > configure:4028: gcc -o conftest     conftest.c  1>&5
> > configure:4051: checking finding CSTACK_ADDRESS
> > configure:4068: gcc -o conftest      conftest.c  1>&5
> > ...
> > 
> > So I don't see a problem here. Do you?
> > 
> > On Thursday, September 15, 2005 5:58 AM Bill Page wrote:
> > > > > 
> > > > > On Thursday, September 15, 2005 5:22 AM I wrote: 
> > > > > 
> > > > > > ... 
> > > > > > Yes, I think you are right. I changed the short name 
> > > > > > (abbreviation=file name?) of my Aldor category from
> > > > > > upper case to lower case and now it **sometimes** works.
> > > > > > See:
> > > > > > 
> > > > > > http://wiki.axiom-developer.org/SandBoxCategoryOfGraphs
> > > > > > 
> > > > > > But often when I try to save this page I get:
> > > > > > 
> > > > > >     >> System error:
> > > > > >     Contiguous blocks exhausted.
> > > > > >   Currently, 1354 pages are allocated.
> > > > > >   Use ALLOCATE-CONTIGUOUS-PAGES to expand the space.
> > > > > > 
> > > 
> > > You can allocate page types explicitly with the #'allocate function
> > > (see the docs if needed),
> > 
> > I do not know what you mean by "allocate page types explicitly".
> > Is this a function like si::allocate-contiguous-pages that I used
> > in the Axiom )lisp command below? Or does this have something to
> > do with the gcl configuration?
> > 
> > > but this is not necessary, as you are totally out of memory here,
> > > and this will just forestall total failure.
> > 
> > As I mentioned in the previous email, all seems well if I call
> > 
> >   )lisp (si::allocate-contiguous-pages 3000 t)
> > 
> > inside Axiom before access the Aldor category. Why do you say
> > that I am "totally out of memory"?
> > 
> > > You need to configure with a greater --enable-maxpage.
> > 
> > That is what I tried to do but the build of gcl fails as
> > I mentioned above.
> > 
> > > 
> > > Do I understand right that someone is layering aldor on top
> > > of axiom?
> > 
> > Aldor used to be available as an optional library compiler
> > for the commercial version of Axiom. The current open source
> > version of Axiom was released without Aldor because NAG had
> > previously and separately given the rights to the Aldor source
> > to "aldor.org" (aka. Stephen Watt). "aldor.org" makes binary
> > versions of Aldor available for free for non-commmercial use
> > but does not follow the current open source licensing model.
> > 
> > Recently Peter Broadbery was able to configure the open source
> > version of Axiom to use the Aldor binary on linux. So now
> > Axiom functions the way it used to in the commercial release
> > but the license status of Aldor is still uncertain. I have made
> > several requests to Stephen Watt to obtain a copy of the
> > source code license for Aldor but I have not yet heard from
> > him.
> > 
> > > Can we distribute such in Debian yet?
> > 
> > I presuemt that to distribute on Debian we would require
> > a GPL-compatible source license to Aldor, right?
> > 
> > > If so, I might be persuaded to package it, and thereby ensure
> > > that it works :-).
> > 
> > Wonderful. I think we should try hard to make this happen.
> > I would love to be able to just 'apt-get install aldor'! :)
> > 
> > Regards,
> > Bill Page.
> > 
> > 
> 
> 
> 
> 
> 

-- 
Camm Maguire                                            address@hidden
==========================================================================
"The earth is but one country, and mankind its citizens."  --  Baha'u'llah