Re: [PATCH] softmmu/physmem: Improve guest memory allocation failure error message

[Dr. David Alan Gilbert]

* David Hildenbrand (david@redhat.com) wrote:
> On 23.08.21 12:34, Philippe Mathieu-Daudé wrote:
> > On 8/23/21 12:24 PM, David Hildenbrand wrote:
> > > On 23.08.21 12:12, Philippe Mathieu-Daudé wrote:
> > > > On 8/23/21 11:29 AM, David Hildenbrand wrote:
> > > > > On 23.08.21 11:23, Peter Maydell wrote:
> > > > > > On Mon, 23 Aug 2021 at 09:40, David Hildenbrand <david@redhat.com>
> > > > > > wrote:
> > > > > > > Not opposed to printing the size, although I doubt that it will 
> > > > > > > really
> > > > > > > stop similar questions/problems getting raised.
> > > > > > 
> > > > > > The case that triggered this was somebody thinking
> > > > > > -m took a byte count, so very likely that an error message
> > > > > > saying "you tried to allocate 38TB" would have made their
> > > > > > mistake clear in a way that just "allocation failed" did not.
> > > > > > It also means that if a future user asks us for help then
> > > > > > we can look at the error message and immediately tell them
> > > > > > the problem, rather than going "hmm, what are all the possible
> > > > > > ways that allocation might have failed" and going off down
> > > > > > rabbitholes like VM overcommit settings...
> > > > > 
> > > > > We've had similar issues recently where Linux memory overcommit 
> > > > > handling
> > > > > rejected the allocation -- and the user was well aware about the 
> > > > > actual
> > > > > size. You won't be able to catch such reports, because people don't
> > > > > understand how Linux memory overcommit handling works or was 
> > > > > configured.
> > > > > 
> > > > > "I have 3 GiB of free memory, why can't I create a 3 GiB VM". "I have 
> > > > > 3
> > > > > GiB of RAM, why can't I create a 3 GiB VM even if it won't make use of
> > > > > all 3 GiB of memory".
> > > > > 
> > > > > Thus my comment, it will only stop very basic usage issues. And I 
> > > > > agree
> > > > > that looking at the error *might* help. It didn't help for the cases I
> > > > > just described, because we need much more system information to make a
> > > > > guess what the user error actually is.
> > > > 
> > > > Is it possible to get the maximal overcommitable amount on Linux?
> > > 
> > > Not reliably I think.
> > > 
> > > In the "always" mode, there is none.
> > > 
> > > In the "guess"/"estimate" mode, the kernel takes a guess (currently
> > > implemented as checking if the mmap size <= total RAM + total SWAP).
> > >      Committable = MemTotal + SwapTotal
> > > 
> > > In the "never" mode:
> > >      Committable = CommitLimit - Committed_AS
> > > However, the value gets further reduced for !root applications by
> > > /proc/sys/vm/admin_reserve_kbytes.
> > > 
> > > Replicating these calculations in user space would be suboptimal IMHO.
> > 
> > What about simply giving a hint about memory overcommit and display
> > a link to documentation with longer description about how to check
> > and figure out this issue?
> 
> That would be highly OS-specific -- for example, there is no memory
> overcommit under Windows. Sure, we could add a Linux specific hint,
> indication documentation. But I'm not sure if most end users stumbling into
> such an error+hint would be able to make sense of memory overcommit details
> (not to mention that they know what it even is) :)
> 
> You can run into memory allocation issues with many applications. Let me
> give you a simple example
> 
> t480s: ~  $ dd if=/dev/zero of=/dev/null ibs=100G
> dd: memory exhausted by input buffer of size 107374182400 bytes (100 GiB)
> 
> So indicating the size of the failing allocation might be just good enough.
> For the other parts it's usually just "the way the OS was configured, it
> does not think it can allow this allocation".

Does it also get complicated by the use of CGroup?

Dave

> -- 
> Thanks,
> 
> David / dhildenb
> 
-- 
Dr. David Alan Gilbert / dgilbert@redhat.com / Manchester, UK