Re: Call for GSoC and Outreachy project ideas for summer 2023

[Eugenio Perez Martin]

On Fri, Jan 27, 2023 at 4:18 PM Stefan Hajnoczi <stefanha@gmail.com> wrote:
>
> Dear QEMU, KVM, and rust-vmm communities,
> QEMU will apply for Google Summer of Code 2023
> (https://summerofcode.withgoogle.com/) and has been accepted into
> Outreachy May 2023 (https://www.outreachy.org/). You can now
> submit internship project ideas for QEMU, KVM, and rust-vmm!
>
> Please reply to this email by February 6th with your project ideas.
>
> If you have experience contributing to QEMU, KVM, or rust-vmm you can
> be a mentor. Mentors support interns as they work on their project. It's a
> great way to give back and you get to work with people who are just
> starting out in open source.
>
> Good project ideas are suitable for remote work by a competent
> programmer who is not yet familiar with the codebase. In
> addition, they are:
> - Well-defined - the scope is clear
> - Self-contained - there are few dependencies
> - Uncontroversial - they are acceptable to the community
> - Incremental - they produce deliverables along the way
>
> Feel free to post ideas even if you are unable to mentor the project.
> It doesn't hurt to share the idea!
>
> I will review project ideas and keep you up-to-date on QEMU's
> acceptance into GSoC.
>
> Internship program details:
> - Paid, remote work open source internships
> - GSoC projects are 175 or 350 hours, Outreachy projects are 30
> hrs/week for 12 weeks
> - Mentored by volunteers from QEMU, KVM, and rust-vmm
> - Mentors typically spend at least 5 hours per week during the coding period
>
> For more background on QEMU internships, check out this video:
> https://www.youtube.com/watch?v=xNVCX7YMUL8
>
> Please let me know if you have any questions!
>
> Stefan
>

Appending the different ideas here.

VIRTIO_F_IN_ORDER feature support for virtio devices
===
This was already a project the last year, and it produced a few series
upstream but was never merged. The previous series are totally useful
to start with, so it's not starting from scratch with them [1]:

Summary
---
Implement VIRTIO_F_IN_ORDER in QEMU and Linux (vhost and virtio drivers)

The VIRTIO specification defines a feature bit (VIRTIO_F_IN_ORDER)
that devices and drivers can negotiate when the device uses
descriptors in the same order in which they were made available by the
driver.

This feature can simplify device and driver implementations and
increase performance. For example, when VIRTIO_F_IN_ORDER is
negotiated, it may be easier to create a batch of buffers and reduce
DMA transactions when the device uses a batch of buffers.

Currently the devices and drivers available in Linux and QEMU do not
support this feature. An implementation is available in DPDK for the
virtio-net driver.

Goals
---
Implement VIRTIO_F_IN_ORDER for a single device/driver in QEMU and
Linux (virtio-net or virtio-serial are good starting points).
Generalize your approach to the common virtio core code for split and
packed virtqueue layouts.
If time allows, support for the packed virtqueue layout can be added
to Linux vhost, QEMU's libvhost-user, and/or QEMU's virtio qtest code.

Shadow Virtqueue missing virtio features
===

Summary
---
Some VirtIO devices like virtio-net have a control virtqueue (CVQ)
that allows them to dynamically change a number of parameters like MAC
or number of active queues. Changes to passthrough devices using vDPA
using CVQ are inherently hard to track if CVQ is handled as
passthrough data queues, because qemu is not aware of that
communication for performance reasons. In this situation, qemu is not
able to migrate these devices, as it is not able to tell the actual
state of the device.

Shadow Virtqueue (SVQ) allows qemu to offer an emulated queue to the
device, effectively forwarding the descriptors of that communication,
tracking the device internal state, and being able to migrate it to a
new destination qemu.

To restore that state in the destination, SVQ is able to send these
messages as regular CVQ commands. The code to understand and parse
virtio-net CVQ commands is already in qemu as part of its emulated
device, but the code to send the some of the new state is not, and
some features are missing. There is already code to restore basic
commands like mac or multiqueue, and it is easy to use it as a
template.

Goals
---
To implement missing virtio-net commands sending:
* VIRTIO_NET_CTRL_RX family, to control receive mode.
* VIRTIO_NET_CTRL_GUEST_OFFLOADS
* VIRTIO_NET_CTRL_VLAN family
* VIRTIO_NET_CTRL_MQ_HASH config
* VIRTIO_NET_CTRL_MQ_RSS config

Shadow Virtqueue performance optimization
===
Summary
---
To perform a virtual machine live migration with an external device to
qemu, qemu needs a way to know which memory the device modifies so it
is able to resend it. Otherwise the guest would resume with invalid /
outdated memory in the destination.

This is especially hard with passthrough hardware devices, as
transports like PCI imposes a few security and performance challenges.
As a method to overcome this for virtio devices, qemu can offer an
emulated virtqueue to the device, called Shadow Virtqueue (SVQ),
instead of allowing the device to communicate directly with the guest.
SVQ will then forward the writes to the guest, being the effective
writer in the guest memory and knowing when a portion of it needs to
be resent.

As this is effectively breaking the passthrough and it adds extra
steps in the communication, this comes with a performance penalty in
some forms: Context switches, more memory reads and writes increasing
cache pressure, etc.

At this moment the SVQ code is not optimized. It cannot forward
buffers in parallel using multiqueue and multithread, and it does not
use posted interrupts to notify the device skipping the host kernel
context switch (doorbells).

The SVQ code requires minimal modifications for the multithreading,
and these are examples of multithreaded devices already like
virtio-blk which can be used as a template-alike. Regarding the posted
interrupts, DPDK is able to use them so that code can also be used as
a template.

Goals
---
* Measure the latest SVQ performance compared to non-SVQ.
* Add multithreading to SVQ, extracting the code from the Big QEMU Lock (BQL).
* Add posted thread capabilities to QEMU, following the model of DPDK to it.

Thanks!

[1] 
https://wiki.qemu.org/Google_Summer_of_Code_2022#VIRTIO_F_IN_ORDER_support_for_virtio_devices