Re: [Qemu-devel] [PATCH v13 5/6] raw-posix: Add full preallocation option

[Kevin Wolf]

Am 04.09.2014 um 15:43 hat Richard W.M. Jones geschrieben:
> On Thu, Sep 04, 2014 at 03:17:51PM +0200, Kevin Wolf wrote:
> > Am 04.09.2014 um 15:07 hat Richard W.M. Jones geschrieben:
> > > On Thu, Sep 04, 2014 at 02:52:57PM +0200, Kevin Wolf wrote:
> > > > Am 04.09.2014 um 14:45 hat Richard W.M. Jones geschrieben:
> > > > > On Thu, Sep 04, 2014 at 02:35:22PM +0200, Kevin Wolf wrote:
> > > > > > Please change the code to always write zeros for FULL,
> > > > > 
> > > > > How is this useful for anyone?  You don't know if the underlying SAN
> > > > > is going to detect these zeroes or combine these blocks together.
> > > > > It's just slow for no reason.
> > > > 
> > > > It's slow for the reason that the user has requested it. Do you doubt
> > > > that users can know what their backend is doing? Why are you insisting
> > > > on providing only the functionality that you personally need?
> > > 
> > > I'm not!  I'm trying to make sure we don't end up with a qemu
> > > interface which is useless for higher layers.  You're proposing
> > > preallocation=full which will be slow but not actually give any
> > > guarantees, or preallocation=meta which is going to be fast but may
> > > not work, and I'm saying that's a dumb interface that's not useful.
> > 
> > So what you propose is an interface that combines both and may
> > unpredictably be slow or fast, and doesn't give any guarantees either.
> > Why would this be any better?
> > 
> > What is your specific use case of full preallocation that wants zero
> > writing, but only implicitly as a fallback? My expectation is that most
> > users want cheap preallocation if it's available, but don't bother to
> > write many gigabytes if it isn't.
> 
> Stepping back, I think what we have are two general approaches:
> 
>  - do the exact thing I want
> 
>  - do the best effort you can
>
> [...]
>
> And it's not just here.  qemu has other places where we'd like "do the
> best thing", not "do the exact thing" ... off the top of my head:
> selecting -cpu, discard support, O_DIRECT.  Libguestfs has to go
> through hoops in these areas, often involving very hairy workarounds
> which reduce reliability.  I'm not saying that more exact options
> aren't also welcome, but doing the best effort is very useful too.

The point is that different people have different ideas of what "do the
best thing" is. You may be surprised to learn that generally we don't
roll a dice to select our defaults, but do it based on what we think is
the best thing overall. And yes, because we have less information than
management tools, it often means that we have to take more conservative
defaults (works for everyone) than what you would want (all the features
that your shiny new guest can use).

> virt-manager, virt-install, virt-clone, libvirt, libguestfs, all
> currently do preallocation with fallback to writing data (via
> hand-written code).  The reason is that customers using simple disks
> (not SANs etc) just want to be sure they're not going to run out of
> space during OS installation.  For the vast majority of users,
> posix_fallocate makes this fast, but people using ext2 or *BSD get the
> slow-but-safe path.

Is this really a concern for many users? I think I never preallocated an
image to make sure it fits. If anything, I run df first, but generally I
assume that it works and if it doesn't, I check what happened.

Basically all users I saw asking for preallocation did it because they
were worried about performance and wanted to get rid of the overhead of
metadata updates during allocations.

> Now it may be that some qemu users are going to want a very exact
> preallocation mode, but that doesn't mean we can't have
> preallocation=besteffort too.

The definition of "besteffort" depends on what you want to achieve. It
is policy, and we generally try to keep policy out of qemu.

It isn't really clear that "make absolutely sure that you never get
-ENOSPC on the file system level, but in the layers below it's okay, and
the resulting performance doesn't matter" is the best thing for everyone.

Kevin