Re: [Qemu-block] [Qemu-devel] storing machine data in qcow images?

[Max Reitz]

On 2018-06-06 13:43, Michal Suchánek wrote:
> On Wed, 6 Jun 2018 13:32:47 +0200
> Max Reitz <address@hidden> wrote:
> 
>> On 2018-06-06 13:19, Michal Suchánek wrote:
>>> On Wed, 6 Jun 2018 13:02:53 +0200
>>> Max Reitz <address@hidden> wrote:
>>>   
>>>> On 2018-06-06 12:32, Michal Suchánek wrote:  
>>>>> On Tue, 29 May 2018 12:14:15 +0200
>>>>> Max Reitz <address@hidden> wrote:

[...]

>>>>>> I hate the idea of making qcow2 a random archive format.    
>>>>>
>>>>> What's wrong with that?    
>>>>
>>>> The fact that qcow2 isn't.
>>>>
>>>> From my perspective it would increase the format's complexity to a
>>>> point where you could just create a new format altogether.  Well,
>>>> actually, all you do is design a filesystem (or reuse an existing
>>>> one).
>>>>  
>>>>>> We have tar for that.    
>>>>>
>>>>> It does not support expanding the stored files.    
>>>>
>>>> Nor does qcow2, because it does not support storing files at all.  
>>>
>>> AFAICT from the previous discussion it already does allow storing
>>> multiple data streams that can be changed independently so it
>>> basically is an archive format or filesystem except the streams are
>>> not named nor easily accessible separately outside of qemu.  
>>
>> I don't quite understand what you are referring to.  We have
>> snapshots, we have bitmaps, yes, but all of that are related directly
>> to the stored guest disk data.
>>
>> The only thing we currently have in qcow2 that is opaque is the VM
>> state that can be stored in snapshots (and don't hold me responsible
>> for that).
> 
> But it has to be related to the stored disk data only because of your
> dislike for stuff not related to the disk data. Not for a technical
> reason. The format can sustain storing unrelated data.

Yes.  Technically, you can even store data without qcow2 knowing, except
that qemu-img check -r leaks will delete it, but yeah.

>>>> Secondly, that completely depends on how you use it.  You can
>>>> freely expand the last file in the archive, for instance.  Also
>>>> I've seen people store files in chunks so they can indeed resize
>>>> it.
>>>>
>>>> (I'm wondering if we could write a block driver that could provide
>>>> such a chunk allocation transparently to qcow2...  Note that a
>>>> qcow2 file does not need to be continuous, so you could in theory
>>>> indeed store the qcow2 file and its data in completely separate
>>>> places in a tar file.)  
>>>
>>> Which basically invents another new filesystem on top of tar for no
>>> good reason. Especially when we have already support for storage
>>> format that is capable enough.  
>>
>> No different from inventing a filesystem on top of qcow2.
>>
>> I don't think qcow2 is any more capable than tar.
> 
> It can natively resize the objects it stores.

It does not store arbitrary objects.  It stores a guest disk (and
snapshots), and bitmaps.  These are stored in directory structures so
new chunks can be dynamically added.  Note that both use slightly
different structures.

Yes, it is technically possible to use a similar structure to store
arbitrary objects.  But currently it does not have that capability.

>                                                tar cannot do that so you
> will have to store random nonsense in the tar archive that only makes
> sense for your fs/tar driver.

Right.  But what's the difference to storing random nonsense in a qcow2
file that not even makes sense to qemu?

>>>> What I'm trying to get at is that qcow2 was not designed to be a
>>>> container format for arbitrary files.  If you want to make it such,
>>>> I'm sure there are existing formats that work better.  
>>>
>>> Such as?  
>>
>> ext2?
> 
> So you want an ext2 driver in qemu instead of expanding qcow2 to work
> not only for a single disk but also for an appliance?

Yes, because ext2 was designed to be a proper filesystem.  I'm not an FS
designer.  Well, not a good one anyway.  So I don't trust myself on
extending qcow2 to be a good FS -- and why would I, when there are
already numerous FS around.

>> It seems to me that you want to make qcow2 a filesystem.  Sure, the FS
>> we'd end up with would probably be simpler than ext2, but I assume
>> thanks to feature creep we'd eventually end up with a qcow2 format
>> that is a worse FS than real FS (especially performance-wise), but
>> that is similarly complex.
> 
> I do not see how the complexity increases drastically by assigning
> user-visible names to some of the data stored in the image. I am not
> familiar with the internals, though.

As I said, qcow2 does not store objects currently.  It stores specific
data structures that differ.  There is no common "object structure".

>>>>>> Unless I have got something terribly wrong (which is indeed a
>>>>>> possibility!), to me this proposal means basically to turn qcow2
>>>>>> into (1) a VM description format for qemu, and (2) to turn it
>>>>>> into an archive format on the way.    
>>>>>
>>>>> And if you go all the way you can store multiple disks along with
>>>>> the VM definition so you can have the whole appliance in one file.
>>>>> It conveniently solves the problem of synchronizing snapshots
>>>>> across multiple disk images and the question where to store the
>>>>> machine state if you want to suspend it.     
>>>>
>>>> Yeah, but why make qcow2 that format?  That's what I completely
>>>> fail to understand.
>>>>
>>>> If you want to have a single VM description file that contains the
>>>> VM configuration and some qcow2/raw/whatever files along with it
>>>> for the guest disk data, sure, go ahead.  But why does the format
>>>> of the whole thing need to be qcow2?  
>>>
>>> Because then qemu can access the disk data from the image directly
>>> without any need for extraction, copying to different file, etc.  
>>
>> This does not explain why it needs to be qcow2.  There is absolutely
>> no reason why you couldn't use qcow2 files in-place inside of another
>> file.
> 
> qemu cannot read the disk data from the file in-place.

Hu?  Why not?

Max

signature.asc
Description: OpenPGP digital signature