Re: [Qemu-devel] [Bug 1769053] Re: Cannot start a guest with more than 1TB of RAM

[ChristianEhrhardt]

On Tue, May 8, 2018 at 10:37 AM, Dr. David Alan Gilbert <address@hidden
> wrote:

> * ChristianEhrhardt (address@hidden) wrote:
> > > Interesting; I thought this was supposed to work.
> >
> > Exactly that was my thought when triaging it initially
> > Furthermore I assume people working la57 (https://lwn.net/Articles/
> 730925/) and such ran tests on much bigger sizes.
>
> I assume so, but I've not looked at the detail of that.
>
> > > Ah right Dan, if you're seeing the 40 bits physical in the guest you
> > definitely need to try the flags I suggest in comment 6; host-phys-
> > bits=true should work for you.
> >
> > I tested Bionic to be at least on libvirt 4.0 / qemu 2.11.1 when we want
> > to check things under the "supposed to work now" flag.
> >
> > Defaults:
> > Host:  address sizes   : 46 bits physical, 48 bits virtual
> > Guest: address sizes   : 40 bits physical, 48 bits virtual
> >
> > I ensured that with option -cpu host,host-phys-bits=true set I
> successfully get what my host can provide in the guest:
> > Guest: address sizes   : 46 bits physical, 48 bits virtual
> >
> > Starting a guest with that >1TB (that would be mostly on swap if needed)
> works just fine as expected. Here ~1063 GB from /proc/meminfo
> > MemTotal:       1114676492 kB
>
> OK, good - that suggests there's nothing missing.
> We enable host-phys-bits=true by default I think (in our machine type?)
>

Interesting approach, I see your comment about that already in [1] when it
was added.
I didn't realize some machine types were setting this already - I assume it
isn't the general default for migratebility to other hosts (like our 36/39
bit laptops).

I assume "we" in this context are RedHat downstream changes to the (some)
machine type(s)?
I see the benefit for huge guests to work without setting those properties,
but I wonder if that caused you trouble in regard to migrations?

[1]: https://patchwork.kernel.org/patch/9223999/


> > I also checked a more compatible approach like -cpu qemu64,phys-bits=42
> > and that works as well.
> >
> > IMHO - if anything - one could argue that libvirt/qemu could be smarter
> > about e.g. auto adding those arguments (or print a warning) when
> > crossing a certain memory size.
>
> The problem is there are a whole bunch of things that are hard to deal
> with:
>   a) Cheaper CPUs tend to have smaller phys-bits even in the same
> generation; e.g. my laptop is still 36 bits, a lot are 39 bits.  I think
> the same is true of the Xeon E3-.... family.   It makes it hard to know
> what to pick when you're going to allow migration.
>
>   b) Reasoning about the total address size range is difficult; you've
> got to take into account PCI address space and hot plug space etc
> to know where the upper edge is.
>

I agree that checking the total address size might have too much false
positives for all the complexities around "estimating" that size.
/me is giving up this idea :-)


> Dave
>
> > So for now I'd stick to the "actually works" summary and keep the status
> > to incomplete.
> >
> > --
> > You received this bug notification because you are subscribed to the bug
> > report.
> > https://bugs.launchpad.net/bugs/1769053
> >
> > Title:
> >   Cannot start a guest with more than 1TB of RAM
> >
> > Status in QEMU:
> >   Incomplete
> > Status in qemu package in Ubuntu:
> >   Incomplete
> >
> > Bug description:
> >   Attempting to start a KVM guest with more than 1TB of RAM fails.
> >
> >   It looks like we might need some extra patches:
> >   https://lists.gnu.org/archive/html/qemu-discuss/2017-12/msg00005.html
> >
> >   ProblemType: Bug
> >   DistroRelease: Ubuntu 18.04
> >   Package: qemu-system-x86 1:2.11+dfsg-1ubuntu7
> >   ProcVersionSignature: Ubuntu 4.15.0-20.21-generic 4.15.17
> >   Uname: Linux 4.15.0-20-generic x86_64
> >   ApportVersion: 2.20.9-0ubuntu7
> >   Architecture: amd64
> >   CurrentDesktop: Unity:Unity7:ubuntu
> >   Date: Fri May  4 16:21:14 2018
> >   InstallationDate: Installed on 2017-04-05 (393 days ago)
> >   InstallationMedia: Ubuntu 16.10 "Yakkety Yak" - Release amd64
> (20161012.2)
> >   MachineType: Dell Inc. XPS 13 9360
> >   ProcKernelCmdLine: BOOT_IMAGE=/vmlinuz-4.15.0-20-generic
> root=/dev/mapper/ubuntu--vg-root ro quiet splash
> transparent_hugepage=madvise vt.handoff=1
> >   SourcePackage: qemu
> >   UpgradeStatus: Upgraded to bionic on 2018-04-30 (3 days ago)
> >   dmi.bios.date: 02/26/2018
> >   dmi.bios.vendor: Dell Inc.
> >   dmi.bios.version: 2.6.2
> >   dmi.board.name: 0PF86Y
> >   dmi.board.vendor: Dell Inc.
> >   dmi.board.version: A00
> >   dmi.chassis.type: 9
> >   dmi.chassis.vendor: Dell Inc.
> >   dmi.modalias: dmi:bvnDellInc.:bvr2.6.2:bd02/26/2018:svnDellInc.:
> pnXPS139360:pvr:rvnDellInc.:rn0PF86Y:rvrA00:cvnDellInc.:ct9:cvr:
> >   dmi.product.family: XPS
> >   dmi.product.name: XPS 13 9360
> >   dmi.sys.vendor: Dell Inc.
> >
> > To manage notifications about this bug go to:
> > https://bugs.launchpad.net/qemu/+bug/1769053/+subscriptions
> --
> Dr. David Alan Gilbert / address@hidden / Manchester, UK
>
> --
> You received this bug notification because you are a member of Ubuntu
> Virtualisation team, which is subscribed to qemu in Ubuntu.
> https://bugs.launchpad.net/bugs/1769053
>
> Title:
>   Cannot start a guest with more than 1TB of RAM
>
> Status in QEMU:
>   Incomplete
> Status in qemu package in Ubuntu:
>   Incomplete
>
> Bug description:
>   Attempting to start a KVM guest with more than 1TB of RAM fails.
>
>   It looks like we might need some extra patches:
>   https://lists.gnu.org/archive/html/qemu-discuss/2017-12/msg00005.html
>
>   ProblemType: Bug
>   DistroRelease: Ubuntu 18.04
>   Package: qemu-system-x86 1:2.11+dfsg-1ubuntu7
>   ProcVersionSignature: Ubuntu 4.15.0-20.21-generic 4.15.17
>   Uname: Linux 4.15.0-20-generic x86_64
>   ApportVersion: 2.20.9-0ubuntu7
>   Architecture: amd64
>   CurrentDesktop: Unity:Unity7:ubuntu
>   Date: Fri May  4 16:21:14 2018
>   InstallationDate: Installed on 2017-04-05 (393 days ago)
>   InstallationMedia: Ubuntu 16.10 "Yakkety Yak" - Release amd64
> (20161012.2)
>   MachineType: Dell Inc. XPS 13 9360
>   ProcKernelCmdLine: BOOT_IMAGE=/vmlinuz-4.15.0-20-generic
> root=/dev/mapper/ubuntu--vg-root ro quiet splash
> transparent_hugepage=madvise vt.handoff=1
>   SourcePackage: qemu
>   UpgradeStatus: Upgraded to bionic on 2018-04-30 (3 days ago)
>   dmi.bios.date: 02/26/2018
>   dmi.bios.vendor: Dell Inc.
>   dmi.bios.version: 2.6.2
>   dmi.board.name: 0PF86Y
>   dmi.board.vendor: Dell Inc.
>   dmi.board.version: A00
>   dmi.chassis.type: 9
>   dmi.chassis.vendor: Dell Inc.
>   dmi.modalias: dmi:bvnDellInc.:bvr2.6.2:bd02/26/2018:svnDellInc.:
> pnXPS139360:pvr:rvnDellInc.:rn0PF86Y:rvrA00:cvnDellInc.:ct9:cvr:
>   dmi.product.family: XPS
>   dmi.product.name: XPS 13 9360
>   dmi.sys.vendor: Dell Inc.
>
> To manage notifications about this bug go to:
> https://bugs.launchpad.net/qemu/+bug/1769053/+subscriptions
>


-- 
Christian Ehrhardt
Software Engineer, Ubuntu Server
Canonical Ltd

-- 
You received this bug notification because you are a member of qemu-
devel-ml, which is subscribed to QEMU.
https://bugs.launchpad.net/bugs/1769053

Title:
  Cannot start a guest with more than 1TB of RAM

Status in QEMU:
  Incomplete
Status in qemu package in Ubuntu:
  Incomplete

Bug description:
  Attempting to start a KVM guest with more than 1TB of RAM fails.

  It looks like we might need some extra patches:
  https://lists.gnu.org/archive/html/qemu-discuss/2017-12/msg00005.html

  ProblemType: Bug
  DistroRelease: Ubuntu 18.04
  Package: qemu-system-x86 1:2.11+dfsg-1ubuntu7
  ProcVersionSignature: Ubuntu 4.15.0-20.21-generic 4.15.17
  Uname: Linux 4.15.0-20-generic x86_64
  ApportVersion: 2.20.9-0ubuntu7
  Architecture: amd64
  CurrentDesktop: Unity:Unity7:ubuntu
  Date: Fri May  4 16:21:14 2018
  InstallationDate: Installed on 2017-04-05 (393 days ago)
  InstallationMedia: Ubuntu 16.10 "Yakkety Yak" - Release amd64 (20161012.2)
  MachineType: Dell Inc. XPS 13 9360
  ProcKernelCmdLine: BOOT_IMAGE=/vmlinuz-4.15.0-20-generic 
root=/dev/mapper/ubuntu--vg-root ro quiet splash transparent_hugepage=madvise 
vt.handoff=1
  SourcePackage: qemu
  UpgradeStatus: Upgraded to bionic on 2018-04-30 (3 days ago)
  dmi.bios.date: 02/26/2018
  dmi.bios.vendor: Dell Inc.
  dmi.bios.version: 2.6.2
  dmi.board.name: 0PF86Y
  dmi.board.vendor: Dell Inc.
  dmi.board.version: A00
  dmi.chassis.type: 9
  dmi.chassis.vendor: Dell Inc.
  dmi.modalias: 
dmi:bvnDellInc.:bvr2.6.2:bd02/26/2018:svnDellInc.:pnXPS139360:pvr:rvnDellInc.:rn0PF86Y:rvrA00:cvnDellInc.:ct9:cvr:
  dmi.product.family: XPS
  dmi.product.name: XPS 13 9360
  dmi.sys.vendor: Dell Inc.

To manage notifications about this bug go to:
https://bugs.launchpad.net/qemu/+bug/1769053/+subscriptions