bug-apl
[Top][All Lists]
Advanced

[Date Prev][Date Next][Thread Prev][Thread Next][Date Index][Thread Index]

Re: Building GNU APL (1.8, SVN: 1252) under OpenBSD/amd64 6.6-STABLE


From: Dr . Jürgen Sauermann
Subject: Re: Building GNU APL (1.8, SVN: 1252) under OpenBSD/amd64 6.6-STABLE
Date: Wed, 8 Apr 2020 17:20:32 +0200
User-agent: Mozilla/5.0 (X11; Linux i686; rv:60.0) Gecko/20100101 Thunderbird/60.6.1

Hi Otto,

thanks, fixed in SVN 1255.

Best Regards,
Jürgen

On 4/8/20 5:05 PM, Dr. Jürgen Sauermann wrote:
Hi Alexander,

thanks for the info. The -l37 option requres that you either:

./configure DYNAMIC_LOG_WANTED=yes

or:

make develop

after ./configure and before make install.

The background of all this is the fact that, by default, GNU/Linux overcommits
memory. It can happen that:

malloc() (or new[] for that matter) returns non-zero (indicating the memory
requested is available) but then the process segfaults when that memory is
being initialized.

In that situation (which is rare, but was indeed observed) the C++ exception
handling fails and the process crashes without a way to catch this case. This
only occurs when memory allocated before is already close to the memory
available. Apparently the exception handler itself is unable to allocate memory
for itself and simply gives up.

In order to avoid this case GNU APL tries to figure the really available memory
in three ways:

1. by analyzing /proc./meminfo which tells how much physical memory might be
available at most. This memory is usually more than you can really get.

2. by analyzing the process memory limit as reported by system call getrlimit().

3. by always limiting the memory to 0xC0000000 (= 3221225472 or about 3Gig
which is one of the numbers that you see in the warning). Apparently
getrlimit()
tells that the process has exactly twice that amount (about 6 Gig which
unfortunately exceeds 32-bit integers. apl -l37 would have given us more
information about the situation.

On GNU/Linux resp. bash 
getrlimit() defaults to 'unlimited' which does not tell
you much unless you set it (e.g. via ulimit -v). However 2. above was created
for those cases where 1. fails (like in your case), and 3. above was meant to deal
with the cases where both 1. and 2. fail.

Now back to the warning: The warning is issued because the memory limit
set for the process exceeds the safety guard set by 3. above. In short the
interpreter believes that the process has permission to use 6 Gig of physical
memory on a machine with only 3 Gig.

The simplest solution for this is to ignore the warning. You will then most likely
get a WS FULL shortly before the actual memory consumption of the process
running the apl interpreter reaches 3 Gig.

Another solution is to suppress the warning by setting a process memory limit
to a value ≤ 3Gig by means of ulimit (hope you have it in BSD).

In neither case will you be able to get a workspace that is bigger that 3Gig.
If that is needed then I have to think about another solution, e.g. a command
line argument and/or a preference setting.

Best Regards,
Jürgen


On 4/8/20 3:39 PM, Alexander Shendi wrote:
Hi Dr. Sauermann,

I'm supplying the desired info:

1. Output of "ulimit -a"

time(cpu-seconds)    unlimited
file(blocks)         unlimited
coredump(blocks)     unlimited
data(kbytes)         6291456
stack(kbytes)        4096
lockedmem(kbytes)    2671438
memory(kbytes)       8010120
nofiles(descriptors) 512
processes            256

2. Contents of /proc/meminfo

OpenBSD does not have a proc-filesystem. I'm assuming that you want
the memory layout. I'm providing info from "dmesg" and "sysctl -a" commands.

2a) Output of dmesg

OpenBSD 6.6 (GENERIC.MP) #7: Thu Mar 12 11:55:22 MDT 2020
    address@hidden:/usr/src/sys/arch/amd64/compile/GENERIC.MP
real mem = 8476545024 (8083MB)
avail mem = 8206925824 (7826MB)
[...]

2b) Output of sysctl -a | fgrep -i mem (edited)

vm.nkmempages=32768
hw.physmem=8476545024
hw.usermem=8386879488
vfs.ffs.dirhash_maxmem=5242880
vfs.ffs.dirhash_mem=3857192

3. Configuration info (Output of "apl --cfg")

*** Warning: the process memory limit (RLIMIT_AS) of 6442450944
 is more than the estimated total_memory of 3221225472.
 This could cause improper WS FULL handling.

configurable options:
---------------------
    ASSERT_LEVEL_WANTED=1 (default)
    SECURITY_LEVEL_WANTED=0 (default)
    APSERVER_PATH=/tmp/GNU-APL/APserver (default)
    APSERVER_PORT=16366 (default)
    APSERVER_TRANSPORT=0 (default)
    CORE_COUNT_WANTED=0  (= sequential) (default)
    DYNAMIC_LOG_WANTED=no (default)
    MAX_RANK_WANTED=8 (default)
    RATIONAL_NUMBERS_WANTED=no (default)
    SHORT_VALUE_LENGTH_WANTED=12, therefore:
        sizeof(Value)       : 456 bytes
        sizeof(Cell)        :  24 bytes
        sizeof(Value header): 168 bytes

    VALUE_CHECK_WANTED=no (default)
    VALUE_HISTORY_WANTED=no (default)
    VF_TRACING_WANTED=no (default)
    VISIBLE_MARKERS_WANTED=no (default)

how ./configure was (probably) called:
--------------------------------------
    ./configure  '--prefix=/home/alexshendi/.local/gnuapl/svn-1254' '--without-postgresql' '--without-gtk3' 'CC=clang' 'CPP=clang -E' 'CXX=clang++'

BUILDTAG:
---------
    Project:        GNU APL
    Version / SVN:  1.8 / 1254M
    Build Date:     2020-04-08 12:20:14 UTC
    Build OS:       OpenBSD 6.6 amd64
    config.status:  '--prefix=/home/alexshendi/.local/gnuapl/svn-1254' '--without-postgresql' '--without-gtk3' 'CC=clang' 'CPP=clang -E' 'CXX=clang++'
    Archive SVN:    1161

4. Output of "apl -137"

This doesn't work (Unknown option '-137'). In case you meant "apl -l 137"
this also doesn't work due to lack of dynamic logging.


I hope this helps. Have a nice day.

Best Regards,

Alexander






reply via email to

[Prev in Thread] Current Thread [Next in Thread]