[Date Prev][Date Next][Thread Prev][Thread Next][Date Index][Thread Index]
Re: [Qemu-devel] storing machine data in qcow images?
From: |
Dr. David Alan Gilbert |
Subject: |
Re: [Qemu-devel] storing machine data in qcow images? |
Date: |
Wed, 6 Jun 2018 10:44:20 +0100 |
User-agent: |
Mutt/1.9.5 (2018-04-13) |
* Eduardo Habkost (address@hidden) wrote:
> On Tue, Jun 05, 2018 at 10:21:59AM +0100, Dr. David Alan Gilbert wrote:
> > <reawakening a fizzled out thread>
> >
> > This seems to have fizzled out because of a lack of a concrete proposal;
> > so here is one based on a reply to Max's post:
> >
> > * Max Reitz (address@hidden) wrote:
> >
> > <snip>
> >
> > > The original problem was that you need to supply a machine type to qemu,
> > > and that multiple common architectures now have multiple machine types
> > > and not necessarily all work with a single image. So far so good, but I
> > > have two issues here already:
> > >
> > > (1) How is qemu supposed to interpret that information? If it's stored
> > > in the image file, I don't see a nice way of retrieving it before the
> > > machine is initialized, at least not with qemu's current architecture.
> >
> > <snip>
> >
> > > (2) Again, I personally just really don't like saving such information
> > > in a disk image. One actual argument I can bring up for that distaste
> > > is this: Suppose, you have multiple images attached to your VM. Now the
> > > VM wants to store the machine type. Where does it go? Into all of
> > > them?
> >
> > <snip>
> >
> > > So I think if we decide to store the machine type, that is kind of a
> > > slippery slope and then there are good arguments for storing even more
> > > configuration options in the file, too. But I really, really don't like
> > > that.
> >
> > <snip>
> >
> > > For another, how do we store the data? key-value seems wrong if we want
> > > to store everything. JSON might be fine. But eventually we just want
> > > basically a qemu configuration file in there, I would think (which may
> > > support JSON at some point?). So basically we would store the data as
> > > a binary blob and let the rest of qemu do its thing with it. But then
> > > please tell me why I fought so valiantly against storing random bitmaps
> > > in qcow2 files. I hate the idea of making qcow2 a random archive
> > > format. We have tar for that.
> >
> > <snip>
> >
> > > tl;dr: I really don't get why it's so hard to supply a config file along
> > > with a qcow2 image. Is it so hard for people to realize that a VM does
> > > not only consist of a disk?
> >
> > Yes! Because in many cases that's all it needs, and it's ready to run
> > with no unpacking.
> >
> > I think we should have:
> >
> > --------------------------------------------------------------
> > Layer 0:
> > QCOW provides a way to store a single string of arbitrary (but
> > limited?) length.
> > QCOW provides a way to replace the string by a new string.
> > The original or the new string will be stored after that;
> > never some mix.
> > Where a file 'b' has a backing file 'a', 'b' inherits the
> > string from 'a' unless 'b' has it's own string.
> > Snapshots inherit their string from the main unless they have
> > their own string.
> >
> > Layer 1:
> > The string shall always be a JSON 'object'; i.e. of the form
> > { "something": ... , "more": ... }
> >
> > The key strings shall be non-null and non-empty and shall
> > be unique.
> >
>
> I'd prefer layer 0+1 to:
>
> 1) Allow multiple entries to be stored (implemented by layer 1
> in this proposal)
> 2) Identify each entry with a name (implemented by layer 1 in
> this proposal)
> 3) Allow arbitrary binary data to be stored on an entry
> (not possible with the JSON-based proposal, because JSON
> strings are not blobs, but Unicode strings).
> 4) Make it easy to replace only one entry while keeping others
> intact (not the case here, if all entries are stored in the
> same JSON string)
>
> I think it would be simpler if layer 0 simply provided a list of
> names/value pairs, where names are ascii strings, and values are
> binary data[1]. It would make layer 1 unnecessary, and allow (3)
> and (4) to happen.
>
> [1] In other words, Rich's proposal of "named blobs":
> https://www.mail-archive.com/address@hidden/msg37856.html
My reasoning was just one of simplicity; each layer in this is
almost trivial.
The downside to my proposal is it's more expensive if you want
to change a single key; but as yet no one has suggested why we'd
want to do it frequently enough to worry.
My suggestion of JSON was just to try and stop the bikeshedding;
we seem to be putting a lot of effort into inventing a new
storage definition for something that so far we need to hold
one rarely changing short string, which even with all the discussion
has moved upto maybe 3 or 4 rarely changing pieces of data.
Dave
>
>
> > Layer 2:
> > '.'s in the key string shall indicate hierarchy
> >
> > Key strings shall be listed in qemu's
> > docs/specs/qcow-keys.rst
> >
> > that shall indicate their meaning and the meaning and
> > valid formatting of the value associated with the,
> >
> > Key strings shall start with either:
> > qemu. in which case they must be listed in a file in
> > the qemu source tree
> >
> > a reverse dotted name unique to the submitter, they may
> > be listed in the same file in the source tree, e.g.
> > com.redhat.
> >
>
> Based on this proposal for layer 2, it looks like you expect the
> number of keys used on layer 1 to become large.
>
> I would prefer a solution that expects a very small set of keys
> for layer 0+1, and point to other specifications of how the blob
> can be interpreted for each key. This way we can experiment with
> different solutions for layers 2-4, instead of deciding on a
> specific format like JSON.
>
>
> > Layer 3:
> > QEMU shall, for a given qcow2 file be able to dump the
> > key values.
> >
> > Layer 4:
> > On creating a VM by importing a qcow2, a management layer
> > shall inspect the key/values to influence the configuration
> > of the VM created. Where it imports multiple qcow2's it
> > shall inspect all the files and flag disagreements.
> >
> > Management layers shall, on creating a qcow2 shall set the
> > keys based on the VM the qcow2 is created for. If the qcow2
> > is created as an additional disk for an exisitng VM it's
> > fine to leave the string empty (e.g. for a data disk).
> >
> > --------------------------------------------------------------
> >
> >
> > Some reasoning:
> > a) I've avoided the problem of when QEMU interprets the value
> > by ignoring it and giving it to management layers at the point
> > of VM import.
> > b) I hate JSON, but there again nailing down a fixed format
> > seems easiest and it makes the job of QCOW easy - a single
> > string.
> > (I would suggest in layer2 that the keys are sorted, but
> > that's a pain to do in some json creators)
> > c) Forcing the registry of keys might avoid silly duplication.
> > We can but hope.
> > d) I've not said it's a libvirt XML file since that seems
> > a bit prescriptive.
> >
> > Some initial suggested keys:
> >
> > "qemu.machine-types": [ "q35", "i440fx" ]
> > "qemu.min-ram-MB": 1024
> >
> >
> > Dave
> >
> > --
> > Dr. David Alan Gilbert / address@hidden / Manchester, UK
> >
>
> --
> Eduardo
--
Dr. David Alan Gilbert / address@hidden / Manchester, UK
- Re: [Qemu-devel] [Qemu-block] storing machine data in qcow images?, (continued)
- Re: [Qemu-devel] [Qemu-block] storing machine data in qcow images?, Eric Blake, 2018/06/05
- Re: [Qemu-devel] [Qemu-block] storing machine data in qcow images?, Richard W.M. Jones, 2018/06/05
- Re: [Qemu-devel] [Qemu-block] storing machine data in qcow images?, Eric Blake, 2018/06/05
- Re: [Qemu-devel] [Qemu-block] storing machine data in qcow images?, Michael S. Tsirkin, 2018/06/05
- Re: [Qemu-devel] [Qemu-block] storing machine data in qcow images?, Eric Blake, 2018/06/05
- Re: [Qemu-devel] [Qemu-block] storing machine data in qcow images?, Michael S. Tsirkin, 2018/06/05
- Re: [Qemu-devel] [Qemu-block] storing machine data in qcow images?, Dr. David Alan Gilbert, 2018/06/06
- Re: [Qemu-devel] [Qemu-block] storing machine data in qcow images?, Gerd Hoffmann, 2018/06/06
- Re: [Qemu-devel] [Qemu-block] storing machine data in qcow images?, Michael S. Tsirkin, 2018/06/05
Re: [Qemu-devel] storing machine data in qcow images?, Gerd Hoffmann, 2018/06/06
Re: [Qemu-devel] storing machine data in qcow images?,
Dr. David Alan Gilbert <=
Re: [Qemu-devel] storing machine data in qcow images?, Max Reitz, 2018/06/06
- Re: [Qemu-devel] storing machine data in qcow images?, Dr. David Alan Gilbert, 2018/06/06
- Re: [Qemu-devel] storing machine data in qcow images?, Max Reitz, 2018/06/06
- Re: [Qemu-devel] storing machine data in qcow images?, Dr. David Alan Gilbert, 2018/06/06
- Re: [Qemu-devel] storing machine data in qcow images?, Max Reitz, 2018/06/06
- Re: [Qemu-devel] storing machine data in qcow images?, Dr. David Alan Gilbert, 2018/06/06
- Re: [Qemu-devel] storing machine data in qcow images?, Daniel P . Berrangé, 2018/06/06
- Re: [Qemu-devel] storing machine data in qcow images?, Dr. David Alan Gilbert, 2018/06/06
- Re: [Qemu-devel] storing machine data in qcow images?, Max Reitz, 2018/06/06