RE: [RFC PATCH 01/12] configure: Add iovisor/ubpf project as a submodule for QEMU

[Zhang, Chen]

> -----Original Message-----
> From: Thomas Huth <thuth@redhat.com>
> Sent: Monday, June 20, 2022 5:44 PM
> To: Zhang, Chen <chen.zhang@intel.com>; Daniel P. Berrangé
> <berrange@redhat.com>
> Cc: Jason Wang <jasowang@redhat.com>; qemu-dev <qemu-
> devel@nongnu.org>; Paolo Bonzini <pbonzini@redhat.com>; Eduardo
> Habkost <eduardo@habkost.net>; Eric Blake <eblake@redhat.com>; Markus
> Armbruster <armbru@redhat.com>; Peter Maydell
> <peter.maydell@linaro.org>; Laurent Vivier <lvivier@redhat.com>; Yuri
> Benditovich <yuri.benditovich@daynix.com>; Andrew Melnychenko
> <andrew@daynix.com>
> Subject: Re: [RFC PATCH 01/12] configure: Add iovisor/ubpf project as a
> submodule for QEMU
> 
> On 20/06/2022 11.29, Zhang, Chen wrote:
> >
> >
> >> -----Original Message-----
> >> From: Thomas Huth <thuth@redhat.com>
> >> Sent: Monday, June 20, 2022 4:47 PM
> >> To: Daniel P. Berrangé <berrange@redhat.com>; Zhang, Chen
> >> <chen.zhang@intel.com>
> >> Cc: Jason Wang <jasowang@redhat.com>; qemu-dev <qemu-
> >> devel@nongnu.org>; Paolo Bonzini <pbonzini@redhat.com>; Eduardo
> >> Habkost <eduardo@habkost.net>; Eric Blake <eblake@redhat.com>;
> Markus
> >> Armbruster <armbru@redhat.com>; Peter Maydell
> >> <peter.maydell@linaro.org>; Laurent Vivier <lvivier@redhat.com>; Yuri
> >> Benditovich <yuri.benditovich@daynix.com>; Andrew Melnychenko
> >> <andrew@daynix.com>
> >> Subject: Re: [RFC PATCH 01/12] configure: Add iovisor/ubpf project as
> >> a submodule for QEMU
> >>
> >> On 20/06/2022 10.11, Daniel P. Berrangé wrote:
> >>> On Mon, Jun 20, 2022 at 05:59:06AM +0000, Zhang, Chen wrote:
> >>>>
> >>>>
> >>>>> -----Original Message-----
> >>>>> From: Daniel P. Berrangé <berrange@redhat.com>
> >>>>> Sent: Friday, June 17, 2022 4:05 PM
> >>>>> To: Zhang, Chen <chen.zhang@intel.com>
> >>>>> Cc: Jason Wang <jasowang@redhat.com>; qemu-dev <qemu-
> >>>>> devel@nongnu.org>; Paolo Bonzini <pbonzini@redhat.com>; Eduardo
> >>>>> Habkost <eduardo@habkost.net>; Eric Blake <eblake@redhat.com>;
> >>>>> Markus Armbruster <armbru@redhat.com>; Peter Maydell
> >>>>> <peter.maydell@linaro.org>; Thomas Huth <thuth@redhat.com>;
> >> Laurent
> >>>>> Vivier <lvivier@redhat.com>; Yuri Benditovich
> >>>>> <yuri.benditovich@daynix.com>; Andrew Melnychenko
> >>>>> <andrew@daynix.com>
> >>>>> Subject: Re: [RFC PATCH 01/12] configure: Add iovisor/ubpf project
> >>>>> as a submodule for QEMU
> >>>>>
> >>>>> On Fri, Jun 17, 2022 at 03:36:19PM +0800, Zhang Chen wrote:
> >>>>>> Make iovisor/ubpf project be a git submodule for QEMU.
> >>>>>> It will auto clone ubpf project when configure QEMU.
> >>>>>
> >>>>> I don't think we need todo this. As it is brand new functionality
> >>>>> we don't have any back compat issues. We should just expect the
> >>>>> distros to ship ubpf if they want their QEMU builds to take advantage
> of it.
> >>>>>
> >>>>
> >>>> Yes, agree. It's the best way to use the uBPF project.
> >>>> But current status is distros(ubuntu, RHEL...) does not ship the
> >>>> iovisor/ubpf like the iovisor/bcc. So I have to do it.
> >>>> Or do you have any better suggestions?
> >>>
> >>> If distros want to support the functionality, they can add packages
> >>> for it IMHO.
> >>
> >> Yes, let's please avoid new submodules. Submodules can sometimes be a
> >> real PITA (e.g. if you forget to update before rsync'ing your code to
> >> a machine that has limited internet access), and if users install
> >> QEMU from sources, they can also install ubpf from sources, too.
> >> And if distros want to support this feature, they can package ubpf on
> >> their own, as Daniel said.
> >
> > Hi Daniel and Thomas,
> >
> > I don't know much the background history of QEMU submodules, but
> meson
> > build is a submodule for QEMU too. It means user can't install QEMU
> > from sources with limited internet access.
> 
> There is no written policy, but I think the general consensus is that we only
> ship code in submodules if:
> 
> 1) It's not available in a required version in distros yet
> 
> and
> 
> 2) it is essentially required to build QEMU (like meson) or if the feature has
> been part of the QEMU sources before and then moved to a separate
> repository (like slirp).
> 
> We ship meson as a submodule since we require some meson features that
> are not available with the meson versions in the distros yet. Once the distros
> catch up, we'll likely remove the meson submodule from QEMU.
> 
> > And back to Daniel's comments,  Yes, the best way is distros add the
> > ubpf packages, But maybe it's too late to implement new features for
> > us. We can introduce the submodule now and auto change to the distros's
> lib when distros add it.  For example QEMU's submodule SLIRP do it in the
> same way.
> 
> slirp used to be part of the QEMU repository, but then has been moved to a
> separate project a while ago. However, at that point in time there weren't
> any packages ins distros yet, so we had to include it as a submodule.
> 
> Now that the distros ship it, too, we're planning to remove the slirp
> submodule from QEMU soon, see:
> 
>   https://lists.gnu.org/archive/html/qemu-devel/2022-04/msg00974.html
> 
> > It make user experience the latest technology with no other
> > dependencies.
> 
> Well, that's only true if we update the submodule in QEMU regularly. If we
> forget to update, we could easily miss some important (maybe even security
> related) fixes from the upstream projects. This can be a nightmare for 
> distros,
> when they then have to go around and look into each and every projects
> whether they embed a certain code module that needs a CVE fix. It's better
> if it can be fixed in one central spot instead.
> 
> > uBPF infrastructure have the ability to extend the capabilities
> > without requiring changing source code. If we not allow it, we have to
> > re-implement all the eBPF assembler, disassembler, interpreter, and JIT
> compiler like DPDK userspace eBPF support (DPDK can't use ubpf project by
> license issue).
> 
> Not sure whether I understood that statement right ... nobody said that
> QEMU should not allow it - we just suggested to rely on a system installation
> of ubpf instead of embedding the code. Or is that not possible?? (I don't
> know that project yet - isn't it possible to compile it as a shared library?)

Thanks for your details explanation.
It looks better to introduce the uBPF shared library for QEMU.
For example:
./configure --ubpf-lib=path

If yes, I think it's fine for me.

Thanks
Chen

> 
>   Thomas