Re: daily: vim problem again

[Jessica Clarke]

On 27 Feb 2021, at 03:57, Paul Dufresne <dufresnep@zoho.com> wrote:
> 
> This looks like one of these moments I think I am brighter than I am... but 
> here my reasoning of the problem so far:
> 
> I am using https://github.com/bminor/glibc/blob/master/malloc/malloc.c
> 
> In it we see:
> 
>> chunk-> +-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+
>> |             Size of previous chunk, if unallocated (P clear)  |
>> +-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+
>> `head:' |             Size of chunk, in bytes                     |A|0|P|
>> mem-> +-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+
>> |             Forward pointer to next chunk in list             |
>> +-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+
>> |             Back pointer to previous chunk in list            |
>> +-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+
>> |             Unused space (may be 0 bytes long)                .
>> .                                                               .
>> .                                                               |
>> nextchunk-> +-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+
>> `foot:' |             Size of chunk, in bytes                           |
>> +-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+
>> |             Size of next chunk, in bytes                |A|0|0|
>> +-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+
>> The P (PREV_INUSE) bit, stored in the unused low-order bit of the
>> chunk size (which is always a multiple of two words), is an in-use
>> bit for the *previous* chunk.  If that bit is *clear*, then the
>> word before the current chunk size contains the previous chunk
>> size, and can be used to find the front of the previous chunk.
>> The very first chunk allocated always has this bit set,
>> preventing access to non-existent (or non-owned) memory. If
>> prev_inuse is set for any given chunk, then you CANNOT determine
>> the size of the previous chunk, and might even get a memory
>> addressing fault when trying to do so.
>> The A (NON_MAIN_ARENA) bit is cleared for chunks on the initial,
>> main arena, described by the main_arena variable.  When additional
>> threads are spawned, each thread receives its own arena (up to a
>> configurable limit, after which arenas are reused for multiple
>> threads), and the chunks in these arenas have the A bit set.  To
>> find the arena for a chunk on such a non-main arena, heap_for_ptr
>> performs a bit mask operation and indirection through the ar_ptr
>> member of the per-heap header heap_info (see arena.c).
>> 
> 
> 
> So Size of chunk, have the 3 lower bits with flags... so to get the size you 
> would shift the value 3 bits to the right, righ?
> 
> But the definition given is:
> 
> #define PREV_INUSE 0x1
> #define IS_MMAPPED 0x2
> #define NON_MAIN_ARENA 0x4
> 
> /*
>   Bits to mask off when extracting size
>   Note: IS_MMAPPED is intentionally not masked off from size field in
>   macros for which mmapped chunks should never be seen. This should
>   cause helpful core dumps to occur if it is tried by accident by
>   people extending or adapting this malloc.
> */
> #define SIZE_BITS (PREV_INUSE | IS_MMAPPED | NON_MAIN_ARENA)
> 
> /* Get size, ignoring use bits */
> #define chunksize(p) (chunksize_nomask (p) & ~(SIZE_BITS))
> 
> /* Like chunksize, but do not mask SIZE_BITS.  */
> #define chunksize_nomask(p)         ((p)->mchunk_size)
> 
> 
> Am I so wrong?

The low 3 bits of the actual size are known to always be 0, so they
aren't stored and instead replaced with the 3 flags. The diagram
doesn't make that overly clear, but if you re-read all the text it
should. Pictorially (not to scale):

+-------------------------+
|       Size in bytes     |
| High bits of size |A|M|P|
+-------------------+-+-+-+

Alternatively, you can imagine that the value stored is indeed missing
3 bits, but it's also stored with 3 extra bits shoved on the end; you
could then recover the original value by ((x >> 3) << 3), but that's
the same as (x & ~7).

Jess