[Date Prev][Date Next][Thread Prev][Thread Next][Date Index][Thread Index]
python asyncio QMP library (AQMP)
From: |
John Snow |
Subject: |
python asyncio QMP library (AQMP) |
Date: |
Thu, 19 Nov 2020 17:00:56 -0500 |
User-agent: |
Mozilla/5.0 (X11; Linux x86_64; rv:78.0) Gecko/20100101 Thunderbird/78.3.1 |
I've been doodling a new Asyncio-based QMP library that might fix some
of the problems[1][2] with our older, non-async QMP library and provide
a better basis for a proper distributable python package for people to
write their own toy scripts to control QEMU.
It's a very early prototype, but I think I have the basics down and
working now. I figured I would float my extremely early copy of it in
case interested parties wanted to take a peek. (Or just to formally
announce that "Hey, I'm working on this!")
https://gitlab.com/jsnow/aqmp/-/tree/devel
The design uses two independent coroutines in the bottom half, a reader
and a writer. execute() works something like this:
+---------+
+---------------- |execute()| <----------+
| +---------+ |
| |
-----------------------------------------------------------
v |
+----+----+ +-----------+ +------+-------+
|Mailboxes| |Event Queue| |Outbound Queue|
+----+----+ +------+----+ +------+-------+
| | ^
v v |
+--+----------------+---+ +-----------+-----------+
| Reader Task/Coroutine | | Writer Task/Coroutine |
+-----------+-----------+ +-----------+-----------+
| ^
v |
+-----+------+ +-----+------+
|StreamReader| |StreamWriter|
+------------+ +------------+
Execute deposits a message in the outbound queue; the writer issues it
to the server. Execute goes to sleep waiting for mail in its inbox. The
reader awakens upon new data available in the stream, the new data is
placed in the mailbox, which awakens the caller.
The Reader/Writer are each executed by _task_runner, which ensures that
any error that occurs in the BH will create a disconnection Task that
will quiesce everything in the bottom half.
Any pending execute actions will be cancelled by the disconnection task;
and will raise "DisconnectedError" to the caller.
Upon learning of a disconnection event, the client code responsible for
managing the overall state of the connection can call disconnect(),
which will raise any exceptions that occurred in the bottom halves that
caused the unscheduled disconnect. From there, the client can decide to
reconnect or abort, depending.
classes and things of interest:
aqmp.py - Mailbox: Simple data structure pairing an asyncio.Event with a
Message. This is done to allow the disconnect task to wake up any
waiting clients even if no message arrived. Python's asyncio.Queue does
not offer a cancel primitive, so this approximates it.
aqmp.py - AQMP: The core of the loop logic is written here; the
public-facing methods are connect(), disconnect(), execute() and
execute_obj(), with a "make_execute_msg" class method available for
creating Messages that can be executed later.
message.py - Message: This class comprises a JSON Object as read off the
wire; putting it in a class like this allows me to write stricter type
guarantees and offer _serialize/_deserialize class methods.
models.py - This module offers strictly typed and validated
deserializations of core QMP structures that are not (generally) defined
by the QAPI schema; i.e., it offers types, de/serialization, and
validation for things such as "Greeting", "SuccessResponse",
"AsynchronousEvent", etc.
What's left to do?
- Work on Exception naming and hierarchy,
more helpful/pretty display of errors.
- Expand the event handling system to be more useful (Maybe?)
- Add QGA support
- Re-add socket creation support (Presently, it relies on the socket
being created already.)
- Documentation, tests, etc.
Might be nice:
- Look into creating a "Server" variant to be used for testing the
library, or even other clients.
- Look into synchronous wrappers that offer thread-safe primitives that
can be used from traditional, synchronous contexts. (preventing "async
creep".)
Long term goal:
- Get it merged in-tree; replace the legacy QMP module over time.
- Create an auto-generated SDK layer based on our QAPI schema that adds
a library of type-safe commands, responses, and events that uses this
QAPI-agnostic QMP library as its core.
- Add my "JobRunner" patches from earlier this year as a full-fledged
jobs API that is designed to make running long-running tasks very
trivial from a python console / qmp-shell.
--js
[1] qmp-shell is not capable of displaying events as they arrive
asynchronously very well.
[2] Our use of non-blocking sockets and socket timeouts in qmp.py relies
quite a bit on undefined behavior that thankfully has not broken yet.
[Prev in Thread] |
Current Thread |
[Next in Thread] |
- python asyncio QMP library (AQMP),
John Snow <=