Re: [Bug-xnee] "WARNING: Enough valuators ... still not printing" - what does it mean?

[L. Rahyen]

> No, not at all. I am, myself, so used to use the stop key to stop recodring 
> (e.g: "-sk q" makes Xnee stop recording when you press 'q') so this is well 
> ..... forgotten, neglected... But it will be fixed.
        Thanks.

> > % ./cnee --record --events-to-record 6 --keyboard
> > ...
> > 7,2,0,0,0,50,0,504416288,20,TypeMatrix.com USB Keyboard
> > 0,2,0,0,0,50,0,504416288
> > 6,2,0,0,0,50,0,504416288,3,Virtual core keyboard
> > 7,3,0,0,0,50,0,504416384,20,TypeMatrix.com USB Keyboard
> > 0,3,0,0,0,50,0,504416384
> > 6,3,0,0,0,50,0,504416384,3,Virtual core keyboard
> > 
> >     To me this looks like unexpected behavior. "0,2,0,0,0,50,0,504416288" 
> > and "0,3,0,0,0,50,0,504416384" contain only duplicate information, and 
> > virtual core events seem to always repeat real device events, so basically 
> > they contain duplicate information too. I naturally expect to see this 
> > output instead:
> > 
> > % ./cnee --record --events-to-record 2 --keyboard
> > ...
> > 7,2,0,0,0,50,0,504416288,20,TypeMatrix.com USB Keyboard
> > 7,3,0,0,0,50,0,504416384,20,TypeMatrix.com USB Keyboard
> > 
> >     When I ask for 6 keyboard events to be recorded, I expect 6 events to 
> > be recorded, not 2. If there is valid reason for current behavior, I would 
> > appreciate if you explain it, perhaps I'm missing something here? This is 
> > true for mouse too. Also, this unnecessary duplication makes log files 
> > about 3 times larger.
> The recording of all the "duplicate" events are done so that Xnee afterwards 
> can replay the session:
>  * as XI master device
>  * as XI slave device
>  * as good old fashioned X device
> I guess we could through in an option to only record XI slave events, or X11 
> core events. But I think the sizes of the log files is not really a major 
> problem so I'd rather not have a new option.
        I think you misunderstood me. What I'm saying is that there is no point 
in recording duplicate information. So there is no need for new option. As you 
see in the example above, I even see no point in saving device id or name for 
virtual core devices. Their ids should not matter when replaying because they 
can be easily discovered at runtime; by the way, cnee already discovers them at 
runtime: virtual core device name or id seems to be never used, so basically 
virtual core events contain as much information as X events as far as cnee is 
concerned - in fact I can simply truncate ",3,Virtual core keyboard" part and 
cnee will replay the log perfectly with --force-core-replay option. And if you 
ignore additional XI information (",20,TypeMatrix.com USB Keyboard" in the 
example above) and replace "7," with "0,", you get an event from "old fashioned 
X device". So if only XI device events are recorded, it is technically possible 
to replay them as XI virtual core events or X core events if necessary. And it 
is technically possible to replay X device events as virtual core events - they 
contain exactly the same information from cnee' point of view anyway (first 
field is the only difference in current format).
        You are right, this is not a major problem. But other problems I have 
reported are not major either because workarounds using simple shell tools can 
be made to filter out "enough valuators" warnings and empty mouse motion 
events, it is possible to write wrapper script to count events correctly, and 
it certainly possible to filter out effectively duplicate lines (and it is 
possible to replay log file without duplicate lines). But workaround is not a 
fix, and writing shell scripts with hacks to get usable/clean/compact 
replayable log does not feel right. I think that when everything just works as 
expected without any workarounds, it is much better.
        Log size is important, because if used as is, it will waste memory: 
both RAM (will use almost 3 times more space in RAM buffer than necessary) and 
disk space. This is especially bad when recording mouse events and RAM buffer 
set to be rarely written to disk (for maximum performance when enough RAM is 
available) because this problem may force it to be written more frequently than 
usually and therefore decrease overall performance. Also, this duplication 
needs to be filtered out if the log needs to be processed by a script. Don't 
get me wrong, I like cnee output because its simple text format makes it easy 
to write a script(s) to process it any way I like, and the fact that each line 
contains all information about each event also can be useful. But obvious data 
duplication like this (of whole lines) is not a good thing, it does not make 
processing simpler (by xnee itself or external script), it just wastes memory 
for no reason.
        However, size and massive data duplication are not the only problems. 
If original XI device is missing or its ID has changed, cnee will crash when 
trying to replay (and if device with same ID but different name will appear it 
couldn't tell the difference).
        I think it would be much better if only events for XI device(s) will be 
recorded (like in my example above) when XI is supported, and if device with 
recorded ID and name does not exist anymore when replay started, try to find 
device with exactly the same name, and if this fails too or if 
--force-core-replay option was given, recorded events should be replayed as 
virtual core events instead, or as X events if no XI support detected or 
--disable-xinput-events was given; if log contains only X events because was 
recorded with --disable-xinput-events (useful to get compact portable log) or 
without XI support at the moment of the recording, replay them as X events by 
default, or as virtual core events if --force-core-replay option was given. 
This way there will be only two types of log: X core events log and XI device 
event log, and both will be interchangeable and portable, only difference 
between the two will be additional two field for XI device ID and name; no 
hackish wrapper scripts will be necessary to get portable and compact XI event 
log, and no crashes will happen if original XI device(s) are missing or their 
ID has changed. Correct me if I'm wrong, but first field seems to be 
unnecessary if it only has 3 possible values, all of which can be autodetected 
by the logic described above. This is of course not only possible solution, but 
it looks like a good one from my point of view.
        All of above is just my opinion, and if you do not have free time to 
fix this particular problem, I understand. I have very little free time myself 
(this is why I report these problems instead of reading XI specs and the xnee 
code and fixing them myself and sending patches). Everybody have different 
priorities and points of view. But if you decide to look into this problem at 
any point in the future, let me know if you need more information.
        I really appreciate the work you already have done. Xnee allowed me to 
relatively easily write a script to record and replay interactive macros 
(interactive in the sense that they can be controlled interactively when they 
are running, unlike simple replay, which cannot be controlled, only stopped), 
allowed me to create some non-interactive automation scripts, and even optimize 
layout on my keyboard by collecting statistics of what I type. I'm also using 
it as sort of backup tool in case I lose something I have typed (I use hotkey 
to pause event recording before typing passwords so there is no security risk) 
- I wrote a script that converts its log to readable format and therefore I can 
easily "restore" something I have typed in the past but lost for whatever 
reason (including browser crash, especially in case when I had typed a lot of 
unsaved text in it). So, as you can see, xnee is very important tool for me 
which I use every day in my scripts and cron jobs. Just thought I share my 
reasons to use xnee in the hope it may be interesting how many things can be 
accomplished with xnee.
        I plan to clean up and improve usability of all my xnee-related scripts 
and make them a part of one project (I do not write its name because I didn't 
register a domain for it yet), add few additional features and GUI front-end, 
and then release it under GPL. Some features I plan to add include 
mouse/touchpad sensitivity calibration and optional mouse/keyboard statistics 
(this feature may remind Mousotron, but more advanced and usable in Linux), in 
both these cases I will need to record XI events, so I can't use 
--disable-xinput-events, and I need to keep logs replayable and portable, 
otherwise some things will not work. This is why I encountered all mentioned 
problems when recording or replaying XI events.     Xnee helped me a lot, 
without it all of this wouldn't be so easy to implement, so I felt that it is 
my duty to at least report all problems/bugs I have found, because if I 
silently create lots of hackish workarounds in my project, no bugs in xnee is 
going to be fixed unless somebody reports them or you encounter them yourself. 
You already fixed some of the bugs I have reported, and I'm very grateful for 
that.

> Think I've found and fixed the bug (Xnee by mistake recorded one of the mouse 
> events when set to record kbd only).
        I have tested it and can confirm that the bug is fixed. Thank you very 
much.