Re: RBD images and exclusive locking

[Ilya Dryomov]

On Thu, Mar 24, 2022 at 3:53 PM Peter Lieven <pl@kamp.de> wrote:
>
> Am 24.03.22 um 12:51 schrieb Peter Lieven:
> > Hi,
> >
> >
> > following the thread on the ceph ml about rbd and exclusive locks I was 
> > wondering if we should rewrite the rbd driver to
> >
> > try to acquire an exclusive lock on open and not magically on the first 
> > write. This way we could directly terminate qemu
> >
> > if the rbd image is in use like it is done with ordinary image files on a 
> > filesystem for some time now.
>
>
> Digging a bit into the code and testing it seems that the exclusive-lock on 
> rbd image is not that exclusive. We can easily start several
>
> qemu instances accessing the same image. Watching the locks on the image, the 
> lock owner of the one exclusive lock
>
> toggles between the instances. This has potentitial for severe corruption.
>
> I am thinking not of the case where a qemu instance gets killed or host dies. 
> I am thinking of network issues where the disconnected
>
> old instance of a VM suddenly kicks back in.
>
>
> However, if I manually acquire an exclusive lock on qemu_rbd_open_image all 
> other callers get EROFS.
>
> So whats the difference between the exclusive lock that a write request 
> obtains and an exclusive lock created
>
> by rbd_lock_acquire? From the rbd lock ls perspective they are identical.

Hi Peter,

I CC'ed you on my reply to Laszlo on ceph-users with some background on
this:

https://lists.ceph.io/hyperkitty/list/ceph-users@ceph.io/thread/MQ6VEXAQTLZ3FL2UAM6W4Z5UVZ5TXJK7/

The manual acquisition with rbd_lock_acquire() is the equivalent of
--exclusive mapping option which disables automatic lock transitions.

Thanks,

                Ilya