Re: [O] Efficiency of Org v. LaTeX v. Word ---LOOK AT THE DATA!

[Colin Baxter]

Dear Christophe,

Great work. You should submit it to http://www.plosone.org/ as a
response. It would be interesting to see what the Referees make of it.


Best wishes,

Colin.

> Hi all,
>
> After seeing Ken's mail:
>
> Le 26/12/2014 23:47, Ken Mankoff a écrit :
>> People here might be interested in a publication from [2014-12-19 Fri]
>> available at http://dx.doi.org/10.1371/journal.pone.0115069
>>
>> Title: An Efficiency Comparison of Document Preparation Systems Used
>> in Academic Research and Development
>>
>> Summary: Word users are more efficient and have less errors than even
>> experienced LaTeX users.
>>
>> Someone here should repeat experiment and add Org into the mix, perhaps
>> Org -> ODT and/or Org -> LaTeX and see if it helps or hurts. I assume
>> Org would trump LaTeX, but would Org -> ODT or Org -> X -> DOCX (via
>> pandoc) beat straight Word?
>>
>>    -k.
>>
>>
> and some of replies it triggered on the list, I went to check the paper. 
> As many of you guys I found some "results" puzzling in particular:
> 1. the use of bar graphs when the data would better be displayed 
> directly (that qualifies immediately the paper as "low quality" for me).
> 2. the larger error bars observed for LaTeX when compared to Word.
> 3. the systematic inverse relationship between the blue and pink bars 
> heights.
>
> So I went to figshare to download the data and looked at them. A quick 
> and dirty "analysis" is attached to this mail in PDF format (generated 
> with org, of course, and this awful software called LaTeX!) and the 
> source org file can be found at the bottom of this mail. I used R to do 
> the figures (and I'm sure the authors of the paper will then criticize 
> me for not using Excel with which everyone knows errors are generated 
> much more efficiently).
>
> I managed to understand the inverse relationship in point 3 above: the 
> authors considered 3 types of mistakes / errors:
> 1. Formatting and typos error.
> 2. Orthographic and grammatical errors.
> 3. Missing words and signs.
> Clearly, following the mail of Tom (Dye) on the list and on the Plos web 
> site, I would argue that formatting errors in LaTeX are bona fide bugs. 
> But the point I want to make is that the third source accounts for 80% 
> of the total errors (what's shown in pink bars in the paper) and clearly 
> the authors counted what the subjects did not have time to type as an 
> error of this type. Said differently, the blue and pink bars are showing 
> systematically the same thing by construction! The second type of error 
> in not a LaTeX issue (and in fact does not differ significantly from the 
> Word case) but an "environment" issue (what spelling corrector had the 
> LaTeX users access to?).
>
> There is another strange thing in the table copy case. For both the 
> expert and novice group in LaTeX, there is one among 10 subjects that 
> did produce 0% of the table but still manage to produce 22 typographic 
> errors!
>
> The overall worst performance of LaTeX users remains to be explained and 
> as mentioned in on the mails in the list, that does not make sense at 
> least for the continuous text exercise. The method section of the paper 
> is too vague but my guess is that some LaTeX users did attempt to 
> reproduce the exact layout of the text they had to copy, something LaTeX 
> is definitely not design to provide quickly.
>
> One more point: how many of you guys could specify their total number of 
> hours of experience with LaTeX (or any other software you are currently 
> using)? That what the subjects of this study had to specify...
>
> Let me know what you think,
>
> Christophe
>
>
----- Snip -----