Re: [Qemu-devel] [PATCH v3 5/5] Add migration stream analyzation script

[Alexander Graf]

-----BEGIN PGP SIGNED MESSAGE-----
Hash: SHA1



On 06.01.15 17:05, Eric Blake wrote:
> On 12/26/2014 07:42 AM, Alexander Graf wrote:
>> This patch adds a python tool to the scripts directory that can
>> read a dumped migration stream if it contains the JSON
>> description of the device states. I constructs a human readable
>> JSON stream out of it.
>> 
>> It's very simple to use:
>> 
>> $ qemu-system-x86_64 (qemu) migrate "exec:cat > mig" $
>> ./scripts/analyze_migration.py -f mig
>> 
>> Signed-off-by: Alexander Graf <address@hidden>
>> 
>> ---
> 
>> diff --git a/scripts/analyze-migration.py
>> b/scripts/analyze-migration.py new file mode 100755 index
>> 0000000..3363172 --- /dev/null +++
>> b/scripts/analyze-migration.py @@ -0,0 +1,592 @@ +#!/usr/bin/env
>> python +# +#  Migration Stream Analyzer +# +#  Copyright (c) 2013
>> Alexander Graf <address@hidden>
> 
> Started in 2013, but you may want to add 2014 and 2015 by the time
> this patch is polished.

Oops :).

> 
>> + +    # The VMSD description is at the end of the file, after
>> EOF. Look for +    # the last NULL byte, then for the beginning
>> brace of JSON.
> 
> Hmm, you picked up on an optimization that I missed in my
> ramblings about 4/5 :)  Since a well-formed JSON description
> contains no NUL bytes or control characters, a pipeline read of a
> migration stream can just continually look for every '\6' marker
> byte followed by '{' as the potential start of an object, and reset
> that search every time a '\0' or another '\6' byte is seen;
> eventually, this will result in just the tail of the file being
> seen as the JSON object (assuming that we never add any other tail
> metadata - or that all other tail metadata is added as 
> backwards-compatible extensions to the JSON).  It STILL might be
> worth the efficiency gain of stashing an offset in the file
> somewhere (as it is faster to seek to an offset than it is to scan
> for particular patterns), but at least your scan works forwards
> rather than in reverse.

Have you tried to run the script yet? Full search, assembly and
analysis of a normal sized migration stream take less than a second -
and that's with in-memory assembly of all the device objects and
properties (which takes the bulk of the time).

I don't think we have an efficiency problem :).

> 
>> + +        # Find the last NULL byte, then the first brace after
>> that. This should +        # be the beginning of our JSON data. +
>> nulpos = data.rfind("\0") +        jsonpos = data.find("{",
>> nulpos)
> 
> Are you sure that there will NEVER be a '{' somewhere between the
> last NUL and the end-of-stream marker?  Shouldn't you really be
> searching for the last QEMU_VM_VMDESCRIPTION ('\6') magic byte,
> rather than NUL?

NUL is the EOF marker, so I'm definitely hoping for it :). If we
decide to add more data after EOF (which we really shouldn't I think),
we can always adjust the script again IMHO.


Alex
-----BEGIN PGP SIGNATURE-----
Version: GnuPG/MacGPG2 v2.0.22 (Darwin)
Comment: GPGTools - http://gpgtools.org

iQIcBAEBAgAGBQJUrFO0AAoJECszeR4D/txg2wUQAIp8R22LiQzqQPrqs5VMiUbE
kOLvieg0fRRHmgQ/SE7aHc/s4sC88EEKsOBintijgcNPkt8Lu65xm71JvI9660BM
9Y/ASYIliNrBKr28ThszK074xAKsvkjSBULIPH6hMUk9e6P8p5SjnKckerWWc7a/
YK7xa+1uxTaK/3+JSzV4ZKFVwbIePrEXqGxYHFcKVqmNs/OSqNwkIkErFJQCyd8D
edqU/zQ4A4v+ZlJl4HZkZqXCBwUGJd6kF0ZUDmF4a7lsAELbv+MTxmVy8enBmH8p
KngnDkI67tpQmlxwSYu9tBW/25qcTIiZOigGT/VShDN6cRQfbw+h3lC3klt/GKA7
ghu464ept6/xI5FQpMkdkPQ742c6lhkuQAUTVqlorVtTsc9ZF2ONLl36Y/xPNmZB
R2BLDq3zDijWe6spWrfoMmc2pmtohHwyBNDtjVbI3nAkhdPddHdR2SzaCUf0B4Nj
tMckMDCucGWYMGTXGMElkw9SZE8kwExPme43nSsGRaHdUVLv24kIv3lNhUlIlpBq
Ljg9CMsHV6RhDySIvucfDueOpmgnyc7KGadXVEFr1pfEJrvg/SjhGDdX+ilmQeRg
iCk0f0/DyJWhPskGDeKBMm9DiJ0QZ3j+Lnw+ZYnOEazyr5tOy0kTuhmPx1eDK9yg
q1YJ7JGn7zVDwoBVWiXK
=wERA
-----END PGP SIGNATURE-----