Re: proper realloc(p,0) behavior?

[Eric Blake]

On 03/25/2011 12:14 PM, Bruno Haible wrote:
> About the question whether a "pointer to an object of size 0" is necessarily
> non-NULL:
> 
> On one hand, you refer to
> http://www.open-std.org/jtc1/sc22/wg14/www/docs/n872.htm
> which refers to 6.2.6.1.(2), which says:
> 
>   "objects are composed of contiguous sequences of one or more bytes"

C99 states in 7.20.3:

"If the size of the space requested is zero, the behavior is
implementation-defined: either a null pointer is returned, or the
behavior is as if the size were some nonzero value, except that the
returned pointer shall not be used to access an object."

A zero-size object is one created by malloc(0), which (necessarily)
points to one or more bytes in memory, but where you may only address
one beyond the end of the zero-length array and where you may not
dereference one beyond the end.  See also gnulib's "zerosize-ptr.h" for
other ways of creating a valid zero-size object according to the C standard.

> 
> On the other hand:
> 
>> calling memchr with size 0 is
>> valid regardless of whether you pass the NULL pointer or any other
>> pointer, because you are not accessing the pointer.

We already argued in the past on this list that calling memchr with a
NULL argument is not valid, because NULL is never a valid object,
zero-size or otherwise.  That's why we changed gnulib's test-memchr to
instead test memchr(zerosize_ptr(),a,0) rather than memchr(NULL,a,0).
However, once you can handle memchr(,0) on any other zero-size object,
most implementations also happen to do what you want for
memchr(NULL,a,0), even though it is technically undefined behavior.

But I don't see how memchr() is relevant to the realloc(p,0) discussion.

>> "If the size of the space requested is zero, the behavior is
>> implementation-defined: either a null pointer is returned, or the
>> behavior is as if the size were some nonzero value, except that the
>> returned pointer shall not be used to access an object."
>> ...
>> AIX malloc(0) is C99 compliant - it takes option two of failing to
>> malloc a zero-sized object.
> 
> This paragraph applies to malloc, realloc, and calloc. If it implies
> that AIX malloc(0) is C99 compliant, then it also implies that it allows
> realloc(p,0) to return NULL in the normal case.

realloc(NULL,0) returning NULL is just as much an error case as
malloc(0) returning NULL, it's just that the error case is distinct from
OOM - if an implementation chooses that malloc(0) can never allocate a
zero-size object (AIX), then NULL represents an EINVAL situation (which
can technically be treated as success); and if an implementation chooses
that malloc(0) should try to allocate a zero-size object (glibc), then a
NULL return represents ENOMEM (which is pretty much always an error).
Either way, when calling realloc(NULL,0), there is no pointer to be
freed in the first place, so on getting a return of NULL, you don't have
to worry about the state of a previous pointer.

But that's not what we're debating about.

We're debating about realloc(p,0) returning NULL, where p is _not_ NULL.
 In that case, C99 requires one of three situations:

1. p is freed _and_ a zero-size pointer is returned (normal BSD
behavior, realloc was a success)
2. a zero-size pointer allocation is attempted but fails due to OOM; so
p is _not_ freed and NULL is returned (BSD behavior, realloc failed)
3. zero-size pointers cannot be allocated by malloc, therefore the
attempt is invalid, and p is _not_ freed and NULL is returned (AIX
behavior, realloc failed)

It does NOT permit glibc behavior:

4. p is freed but NULL is returned

because NULL is reserved for errors, but freeing p without allocating a
zero-size pointer, especially when malloc(0) or realloc(NULL,0) will
allocate a zero-size pointer, is a success case.


The whole thing that sparked this debate (now raging in the Austin
Group, in the C99 committee, in the glibc bug report, and now in
bug-gnulib), was my desire to make the next revision of POSIX impose
tighter requirements on malloc to make it possible to expose the choice
between the implementation-defined options to the program, thus making
it possible to tell if malloc(0) returning NULL is a success
(implementation defines that it always returns NULL) or a failure
(implementation defines that it tries to return a zero-size pointer, but
ran out of memory).  That is, I'd like for one of these two scenarios:

errno = 0;
if (!(p = malloc(0)))
  if (errno)
    ...; // malloc(0) tried to do zero-size object, but failed
  else
    ...; // malloc(0) always successfully returns NULL

or:

if (!(p = malloc(0)))
  if (errno == EINVAL)
    ...; // malloc(0) always returns NULL
  else
    ...; // malloc(0) tried to do zero-size object, but failed

However, since AIX implements the first option (errno unchanged) rather
than the second (errno is set to EINVAL), the first option is more
likely to gain any traction for standardization.


Everything in my proposal went fine for malloc(0) and calloc(0), and
even for realloc(NULL,0).  It wasn't until we started discussing
realloc(p,0) that it became questionable on how things should behave:

Should realloc(p,0) return a zero-size object?  If malloc(0) always
returns NULL, then the answer is obvious: no.  And in that case, the
value of errno would be the same as for malloc(0) (either unchanged
implying success, or EINVAL implying not possible).

If malloc(0) normally returns a zero-size object, then the behavior of
realloc(p,0) is less clear.  If it returns a zero-size object, then
realloc was obviously successful, and p was freed; but now you have to
free that zero-size object or suffer from a leak.  But the C89 wording
(which is still in POSIX) effectively stated that the operation was
equivalent to free(p), and returning a new object defeats that
equivalence (what's the point in using calling realloc(p,0) to free a
pointer if you just get another pointer that you have to free later?).
Hence the GNU behavior of always returning NULL after freeing p.  On the
other hand, when you consider that the C99 wording no longer talks about
realloc(p,0) being equivalent to free(p), but mandates a reallocation to
a zero-size object, then it totally makes sense to return a new object
(realloc is not for freeing objects, but for changing their size; use
free(p) if you wanted to free it without replacement).

On the other hand, when realloc(NULL,0) normally returns a zero-size
object, then if realloc(p,0) returns NULL, then how can you tell if the
operation was a success (and p was freed) or a failure (a zero-size
allocation was attempted but no memory was available, so p is unchanged)?

Since C99 unfortunately does not discuss the effects of realloc on
errno, there is no way to tell implementations apart.  The other wrinkle
is that while the POSIX wording appears to permit the GNU behavior,
POSIX also states that any conflict with C99 is unintentional and that
C99 should trump.

It may be that the end game solution is that POSIX specifically states
that realloc(p,0) frees p even while returning NULL, by using errno as
the key to distinguish the difference (C99 can't do that, because C99
doesn't state any requirements on errno).  But that bothers the C99
committee because it then means that you can't write a program that is
portable to C99 but still runs in POSIX (that is, if you assume C99
semantics but have GNU behavior and POSIX condoned GNU behavior, you
will get a double free).

My personal take is that the C99 committee has already answered that
they do _not_ want to permit GNU behavior, and that Uli has already
answered that he does not want to switch to C99 behavior, so the only
sane thing left for POSIX to do without invalidating glibc would be to
require that portable applications shall not call realloc(p,0) for
non-NULL p, thereby defining away the problem in POSIX, and leaving it
up to glibc whether complying with POSIX is sufficient, or whether it is
also worth complying with C99 (although it would be the first case where
being POSIX compliant no longer implies C99 compliance).

-- 
Eric Blake   address@hidden    +1-801-349-2682
Libvirt virtualization library http://libvirt.org

signature.asc
Description: OpenPGP digital signature