On 5/24/21 9:32 AM, Stefan Hajnoczi wrote:
On Sat, May 22, 2021 at 12:32:00AM +0530, Niteesh G. S. wrote:
Welcome Niteesh :) I look forward to working with you this summer.
By end of this summer, I would like to get a basic TUI with some desirable
features working. Some of the features I would like to get working are
As a reminder to anyone reading this thread, the goal is to create a qmp-shell
that functions more as a TUI, akin to mutt, irssi, or (my favorite example)
mitmproxy. The idea is that there will be, at minimum, a history panel showing
QMP messages that have occurred so far and a text entry panel for entering new
commands.
This shell can then be augmented with various other features to facilitate testing,
debugging, etc. One of the core upgrades over the existing qmp-shell will be the
featuring of truly asynchronous events which will appear in the history panel without
requiring the human user to press <enter> to allow them to display. This will
use a new Asynchronous QMP library to facilitate this feature, bringing with it fixes
over our current use of undocumented Python features abusing non-blocking sockets in
the old QMP library.
My plan is to worry about implementing the very basics of the shell first, and
then to add more features on as we feel comfortable with the basics. We can
discuss what we consider to be the bare minimum for this project and lay out
the feature requirements that define a successful minimum.
1) Syntax checking
To a limited extent. I don't want to disallow the user from sending commands
that are invalid in the event that we want to test the server's ability to cope
with and reply to invalid commands.
However, if the syntax is malformed enough that we can't understand it to send
it to the server, good error messages that point out what exactly went wrong
are helpful.
2) Syntax highlighting
3) Code completion
4) Logging
I would like to hear some of the features you would like to have and also
your
advice's on implementation.
Welcome Niteesh!
It would be great to have an QMP API documentation viewer built into the
TUI. When you enter a command like 'migrate' it would automatically
display the associated QMP command documentation from qapi/*.json.
Stefan's suggestion is an important one to me. The current qmp-shell does have an
auto-complete list of available commands (press <tab> after connecting to see
the list), but it offers no explanation of what the commands do, what their arguments
are, etc.
Being able to get interactive help explaining the commands, their arguments,
etc would be a massive usability improvement over the old shell.
Stefan
Additionally, there's a few threads we've had before with people's stretch
goals, dreams, nice-to-haves and so on. I owe you a bit of a compilation of
those ideas. Here are some that I happen to remember.
(Keep in mind that not all of these have been discussed or vetted thoroughly on
list, some may be vastly harder than others, and not all of them may be
mutually compatible. There may even be objections to some of these. We will
definitely not do every last thing on this list, and it is important we stay
focused on the core task and get code merged this summer. However, some of
these stretch goals might be fun and provide you with extreme leeway to
investigate features that interest you personally, so I am listing them now.)
1. Utilize or otherwise re-implement readline's history feature, such that when
opening qmp-shell, you can press up/down to recall previously issued commands.
(The usage of 'readline' library itself may be complicated if we wish to use a
TUI library like urwid. Some re-creation of features may be necessary.)
2. Implement tab-complete for command names, command arguments, etc.
3. Implement a FiSH-like suggestion mode where, without tab complete, the text entry
window shows (in e.g. a gray font) the most likely auto-complete. You can press
<tab> to accept this auto-completion.
4. Add the ability to save a log file for a given session, recording
timestamps, incoming messages and outgoing messages. Sessions could possibly be
automatically saved and cycled such that the last 10 or so are kept, or maybe
up 10MB, or some other kind of limit.
5. Add the ability to load an old log file and "replay" it to a server. Optionally, the
"replay" might be able to use the log timestamps to replay the commands with an identical
timing. After a replay, it might be nice to be able to use a diff viewer of some kind to show
differences between the original script and the replay.
6. Add the ability to "attach" qmp-shell to an existing instance of a libvirt-managed
QEMU and have qmp-shell behave in a "watch" mode, showing commands sent back and forth
between QEMU and libvirt, to help facilitate live debugging of libvirt.
7. Add the ability to "hide" certain events from the history view. By pressing
a certain hotkey, we can enter a command mode and start filtering certain events. For
example, we may wish to hide from view all JOB_STATUS_CHANGE events from view. Certain
filtering modes may warrant their own explicit mode available directly in the shell, but
we may wish to provide a more powerful filtering/scripting mechanism.
8. Similar to above, add the ability to change the "verbosity" of events in the
history panel live at runtime. For example:
- Show the full QMP message, pretty-printed
- Show the full QMP message, condensed with no spaces.
- Show the raw byte sequence of each QMP message.
- Show an abbreviated summary; just the command name or event name.
- Show or hide timestamps
- Show outgoing commands in terms of what the user typed in qmp-shell.
(e.g., if we offer a simple syntax, show the simple-syntax version.)
Being able to toggle between these at-will may be nice to help see
more or less information dynamically as the situation calls for it.
9. The ability to generate an iotest stub from the current session history or a
subsection thereof. It's OK if the test isn't fully functional on its own,
leaving some sections as stubs. Generating the commands, waiting for the
commands to complete, and generating an '.out' could be a nice feature that
helps qmp-shell users quickly write tests.
Some thoughts on this:
- iotests usually launch a VM to connect to. qmp-shell will not know the
arguments QEMU was launched with. The generated test might attempt to launch
QEMU with a minimum viable configuration, leaving other arguments as a stub
marked #FIXME.
- The generated iotest may be able to automatically generate event waits. For
instance, if a JOB_STATUS_CHANGE event occurs prior to the user entering the
next command, the generated iotest could automatically insert a delay that
waits for the same event.
- Generated tests should always end with a VM shutdown sequence, either
explicitly (user sends a 'quit' command) or implicitly (The python iotest
library calls vm.shutdown())
10. The ability to detect a disconnection and attempt to reconnect
automatically, similar to how an IRC client would.
11. The ability to define macros to perform multi-step, complex routines with a
single command. Something like a ~/.config/qemu/qmp-shell.conf file could
possibly define these macros. The ability to create/save/edit these macros
directly from the shell could be nice.
12. Integrating tools like Vladimir's block graph visualization tool. There are
complex memory structures in QEMU that can be hard to understand quickly by
reading large volumes of text. qmp-shell could be augmented to help render and
visualize them quickly, facilitating debugging greatly.