Re: [ANNOUNCE] libblkio v0.1.0 preview release

[Daniel P . Berrangé]

On Thu, Apr 29, 2021 at 04:00:38PM +0100, Richard W.M. Jones wrote:
> On Thu, Apr 29, 2021 at 03:41:29PM +0100, Stefan Hajnoczi wrote:
> > On Thu, Apr 29, 2021 at 03:22:59PM +0100, Richard W.M. Jones wrote:
> > > On Thu, Apr 29, 2021 at 03:05:50PM +0100, Stefan Hajnoczi wrote:
> > > > The purpose of this preview release is to discuss both the API design
> > > > and general direction of the project. API documentation is available
> > > > here:
> > > > 
> > > >   https://gitlab.com/libblkio/libblkio/-/blob/v0.1.0/docs/blkio.rst
> > > 
> > > libvirt originally, and now libnbd, keep a per-thread error message
> > > (stored in thread-local storage).  It's a lot nicer than having to
> > > pass &errmsg to every function.  You can just write:
> > > 
> > >  if (nbd_connect_tcp (nbd, "remote", "nbd") == -1) {
> > >    fprintf (stderr,
> > >             "failed to connect to remote server: %s (errno = %d)\n",
> > >             nbd_get_error (), nbd_get_errno ());
> > >    exit (EXIT_FAILURE);
> > >  }
> > > 
> > > (https://libguestfs.org/libnbd.3.html#ERROR-HANDLING)
> > > 
> > > It means you can extend the range of error information available in
> > > future.  Also you can return a 'const char *' and the application
> > > doesn't have to worry about lifetimes, at least in the common case.
> > 
> > Thanks for sharing the idea, I think it would work well for libblkio
> > too.
> > 
> > Do you ignore the dlclose(3) memory leak?
> 
> I believe this mechanism in libnbd ensures there is no leak in the
> ordinary shared library (not dlopen/dlclose) case:
> 
> https://gitlab.com/nbdkit/libnbd/-/blob/f9257a9fdc68706a4079deb4ced61e1d468f28d6/lib/errors.c#L35
> 
> However I'm not sure what happens in the dlopen case, so I'm going to
> test that out now ...

pthread_key destructors are a disaster waiting to happen with
dlclose, if the dlclose happens while non-main threads are
still running. When those threads exit, they'll run the
destructor which points to a function that is no longer
resident in memory.

IOW if you have destrutors, then you need to make sure your
library uses "-z nodelete" when linking, to turn dlclose()
into a no-op.

  commit 8e44e5593eb9b89fbc0b54fde15f130707a0d81e
  Author: Daniel P. Berrangé <berrange@redhat.com>
  Date:   Thu Sep 1 17:57:06 2011 +0100

    Prevent crash from dlclose() of libvirt.so
    
    When libvirt calls virInitialize it creates a thread local
    for the virErrorPtr storage, and registers a callback to
    cleanup memory when a thread exits. When libvirt is dlclose()d
    or otherwise made non-resident, the callback function is
    removed from memory, but the thread local may still exist
    and if a thread later exists, it will invoke the callback
    and SEGV. There may also be other thread locals with callbacks
    pointing to libvirt code, so it is in general never safe to
    unload libvirt.so from memory once initialized.
    
    To allow dlclose() to succeed, but keep libvirt.so resident
    in memory, link with '-z nodelete'. This issue was first
    found with the libvirt CIM provider, but can potentially
    hit many of the dynamic language bindings which all ultimately
    involve dlopen() in some way, either on libvirt.so itself,
    or on the glue code for the binding which in turns links
    to libvirt



Regards,
Daniel
-- 
|: https://berrange.com      -o-    https://www.flickr.com/photos/dberrange :|
|: https://libvirt.org         -o-            https://fstop138.berrange.com :|
|: https://entangle-photo.org    -o-    https://www.instagram.com/dberrange :|