Re: [Qemu-devel] [PATCH] hostmem-file: add a property 'notrunc' to avoid data corruption

[Igor Mammedov]

On Thu, 20 Oct 2016 14:59:11 -0200
Eduardo Habkost <address@hidden> wrote:

> On Thu, Oct 20, 2016 at 05:41:47PM +0200, Igor Mammedov wrote:
> > On Thu, 20 Oct 2016 13:15:31 -0200
> > Eduardo Habkost <address@hidden> wrote:
> >   
> > > On Thu, Oct 20, 2016 at 04:17:35PM +0200, Igor Mammedov wrote:  
> > > > On Thu, 20 Oct 2016 11:47:18 -0200
> > > > Eduardo Habkost <address@hidden> wrote:
> > > >     
> > > > > On Thu, Oct 20, 2016 at 09:33:53PM +0800, Haozhong Zhang wrote:    
> > > > > > On 10/20/16 11:21 -0200, Eduardo Habkost wrote:      
> > > > > > > On Thu, Oct 20, 2016 at 02:34:12PM +0200, Igor Mammedov wrote:    
> > > > > > >   
> > > > > > > > On Thu, 20 Oct 2016 14:13:01 +0800
> > > > > > > > Haozhong Zhang <address@hidden> wrote:
> > > > > > > >       
> > > > > > > > > If a file is used as the backend of memory-backend-file and 
> > > > > > > > > its size is
> > > > > > > > > not identical to the property 'size', the file will be 
> > > > > > > > > truncated. For a
> > > > > > > > > file used as the backend of vNVDIMM, its data is expected to 
> > > > > > > > > be
> > > > > > > > > persistent and the truncation may corrupt the existing data.  
> > > > > > > > >     
> > > > > > > > I wonder if it's possible just skip 'size' property in your 
> > > > > > > > case instead
> > > > > > > > 'notrunc' property. That way if size is not present one'd get 
> > > > > > > > actual size
> > > > > > > > using get_file_size() and set 'size' to it?
> > > > > > > > And if 'size' is provided and 'size' != file_size then error 
> > > > > > > > out.      
> > > > > > > 
> > > > > > > I think it is valid to start with a zero-size file and then let
> > > > > > > QEMU extend it.      
> > > > > > 
> > > > > > For vNVDIMM, extending from zero-size file can be valid when a file 
> > > > > > is
> > > > > > first used. However, it's not valid for the second and following use
> > > > > > of the same file.
> > > > > >       
> > > > > > > But I agree we should: 1) make 'size' optional as
> > > > > > > you suggested; 2) never truncate the file to a smaller size.
> > > > > > >       
> > > > > > 
> > > > > > I will add another patch for this. Is there any way in QEMU to 
> > > > > > decide
> > > > > > whether a memory-backend-file object is used for vNVDIMM when the
> > > > > > object is being created? Or 'size' can be optional for all kinds of
> > > > > > usages?      
> > > > > 
> > > > > I believe 'size' can be optional for all usage, because at the
> > > > > moment the memory allocation code asks the backend for a memory
> > > > > region, it is supposed to know desired RAM size from the frontend
> > > > > configuration (-numa, -m, or "size" property of pc-dimm).    
> > > > 
> > > > nope, currently the size propagates other way around
> > > >  from back-end to front-end and not backwards    
> > > 
> > > I'd say that this is a bug. Frontend size is guest ABI and
> > > shouldn't be overridden by backend configuration if it's
> > > explicitly set.  
> > frontend.size is always <= backend.size
> > 
> > allocation specified when backend is created (-object/object_add)
> > and front end size if needed/used is <= backend size
> > 
> > so far code followed this design.  
> 
> This would work, but it doesn't happen in the case of -numa:
> 
>   $ ./x86_64-softmmu/qemu-system-x86_64 \
>     -object memory-backend-file,id=mem0,mem-path=/tmp/mempath,size=2G \
>     -numa node,size=1G,memdev=mem0 -m 1G
>   qemu-system-x86_64: total memory for NUMA nodes (0x80000000) should equal 
> RAM size (0x40000000)
> 
> That's a bug we need to fix.
it's mismatch between option "-m 1G" and sum of memory provided by options
"-numa node,memdev=mem0 -object 
memory-backend-file,id=mem0,mem-path=/tmp/mempath,size=2G"

Adding -numa 'size=1G' is a bug in above example as it's not supported option,
but parse_numa somehow silently ignores it instead of failing.

Allowing to map only 1G RAM of provided by backend 2G is kind of
nonsense. Currently for pc-dim devices we allow 1:1 mapping only
and initial memory falls into the same category, if we consider
conversion of initial memory to set of pc-dimm devices then we
shouldn't allow to do partial mapping for '-numa memdev' either.
if we convert current numa memory mapping it pc-dimm terms it
would look like:
  -object memory-backend-file,id=mem0,mem-path=/tmp/mempath,size=G
  -device pc-dimm,memdev=mem0,node=0
  ...
and we could drop/obsolete "-m SIZE -numa [mem|memdev]... CLI options