Re: [help-recutils] Recutils usage examples and performance questions

[Jose E. Marchesi]

Hi Advrk.
    
    I recently stumbled upon recutils and am very interested in using it,
    but have a few questions:
    
    (1) How active is development and is it well supported? Is there a
    roadmap for new features?

Well, I fix bugs as they get reported, and release a new version each
year.  So I guess the answer is yes: recutils is actively developed and
supported :)

Regarding the roadmap, I have several ideas, but they are not documented
anywhere.  Feel free to share with us any interesting idea you may have.
    
    (2) The bundled CLI utilities seem easy to use and useful, a big
    attraction for me. However I don't know C and only a bit of Python,
    and there is a Python implementation
    (https://github.com/martin-damien/pyrecutils), but it seems to be
    dead.... Are there any other bindings or ports out there?

During the last Google Summer of Code a student wrote python bindings
for librec.  However, these patches still have to be merged.
    
    (3) I love recutils' simplicity and human-readable-text-based
    approach. The documentation says "recfiles can be used to store
    medium-sized databases". How big is that? A recfile of 100s, 1000s, or
    more records? How's the performance?

It works well for my needs.  Most operations with thousand of registers
work with no noticeable delay.

    (4) Are there real world usage cases/examples of recutils that
    illustrate how people are deploying it?
    
I can refer to two examples.

We use recfiles to administer GNUs participation in the Google Summer of
Code program.  See http://www.gnu.org/s/soc-projects/soc2014.rec.

For a more complex example, I can show you the record descriptors of a
database I maintain internally at my job.  I'm sorry I can't give you
the whole file as it contains confidential info:

%rec: Instruction
%key: Name
%mandatory: Function ISA Class Op Assembly
%type: Function line
%type: ISA enum
+ V7 V8 V9
+ VIS1 VIS2 VIS2PLUS VIS3 VIS3B (subset of VIS3 implemented in SPARC64X+) VIS4
+ FMAF IMA OSA2011
+ CRYPTO
%type: Flag enum
+ D    (deprecated instruction)
+ H    (hyperprivileged instruction)
+ Hdis (Privileged action if in nonprivileged or privileged mode and access is 
disabled)
+ N    (non-portable instruction)
+ P    (privileged instruction)
+ Pasi (privileged if bit 7 of ASI is 0)
+ Pasr (privileted if referred ASR register is privileged)
+ Pdis (privileged if in nonprivileged mode and nonprivileged mode is disabled)
# The class of an instruction is composed by a big letter and a digit, the
# meaning of these being:
#
#        Letter                  Digit
#        ------                  -----
#    A: Use freely.         1: Native fast
#    B: Use carefully.      2: Native slow
#    C: New feature.        3: Trap and emulate
#    D: Deprecated.         4: Trap and die
#    N: Non-portable.       5: Reused
#    P: Private.
#    Y: Don't use.
#    Z: Can't use.
# 
# Refer to OSA2011 for a deeper explanation on what these
# categories mean.
%type: Class regexp /[ABCDNPYZ][12345]/
%type: Exception enum
+ illegal_instruction fp_disabled compatibility_feature privileged_opcode
+ control_transfer_instruction mem_address_not_aligned
+ LDDF_mem_address_not_aligned LDQF_mem_address_not_aligned privileged_action
+ VA_watchpoint DAE_invalid_asi DAE_privilege_violation DAE_nc_page
+ DAE_nfo_page DAE_nfo_violation DAE_side_effect_page fast_data_access_MMU_miss
+ data_access_MMU_miss data_access_MMU_error fast_data_access_protection
+ PA_watchpoint data_access_error fp_exception_other fp_exception_ieee_754
+ spill_n_normal spill_n_other unimplemented_LDTW umplimplemented_STTW
+ fill_n_normal fill_n_other division_by_zero software_initiated_reset
+ clean_window STDF_mem_address_not_aligned STQF_mem_address_not_aligned
+ trap_instruction unimplemented_STTW tag_overflow htrap_instruction
# Types for intruction bit fields.  Single white characters are used
# in some types in order to improve readability (i.e. to facilitate
# mental translation to hexadecimal or octal numbers) or to resemble
# their disposition in the actual instructions.
%type: Op                regexp /^[01]{2}$/
%type: Op2,RCond         regexp /^[01]{3}$/
%type: Op3               regexp /^[01]{2} [01]{4}$/
%type: Op5               regexp /^[01]{2} [01]{2}$/
%type: Opf               regexp /^[01] [01]{4} [01]{4}$/
%type: OpfLow,Fcn,CbCond regexp /^[01] [01]{4}$/
%type: Cond              regexp /^[01]{4}$/
%type: Bit28             regexp /^[01]$/
%type: Rs1,Rs2,Rd        regexp /^[01] [01]{4}$/
%type: Imm22             regexp /^[01]{2} [01]{4} [01]{4} [01]{4} [01]{4} 
[01]{4}$/
%type: X                 regexp /^[01]$/
# The following constraints reflect several dependency relationships
# which exist among bit fields in some instructions.
%constraint: Op = "10"                    => #Op3
%constraint: Op = "11"                    => #Op3
%constraint: Op = "10" && Op3 = "11 0110" => #Opf
%constraint: Op = "10" && Op3 = "11 0101" => #Opf || (#OpfLow && #RCond)
%constraint: Op = "10" && Op3 = "10 1111" => #RCond
%constraint: Op = "10" && Op3 = "10 1100" => #Cond
%constraint: Op = "10" && Op3 = "11 0100" => #Opf
%constraint: Op = "10" && Op3 = "01 1001" => #Op5
%constraint: Op = "10" && Op3 = "11 0111" => #Op5
%constraint: Op = "10" && Op3 = "11 0001" => #Fcn
%constraint: Op = "10" && Op3 = "11 0000" => #X || #Rd
%constraint: Op = "00" && Op2 == "011"    => #Bit28
%constraint: Op = "00" && Op2 = "011" && Bit28 = "1" => #CbCond
# Instructions can update the value of one or more registers
# as a side effect, i.e. not the Rd register.  We use the
# field `Updates' to denote this.
%type: Updates rec Register

[...]

%rec: Register
%key: Name
%mandatory: Type
%type: Type enum
+ GPR (General-Purpose register)
+ FPR (Floating-Point register)
+ CR  (Control-Register)
+ ASR (Ancillary State Register)
# The canonical names of the registers are constrained to its
# type.
%constraint: Type = "GPR" => Name ~ "^R[0-9]+$"
%constraint: Type = "FPR" => Name ~ "^F[SDQ][0-9]+$"
# A register can be read-only, write-only or read/write.
%type: Perms enum
+ RD   (read-only)
+ WR   (write-only)
+ RDWR (read-write)
%mandatory: Perms
# An access to a register (read or write) can be of three privilege
# levels: unprivileged, privileged or hyperprivileged.
%type: Level,ReadLevel,WriteLevel enum
+ U (unprivileged)
+ P (privileged)
+ H (hyperprivileged)
%constraint: #Level || #ReadLevel || #WriteLevel
# The width of the registers are expressed in bits.
%mandatory: Width
%type: Width int
# Many floating-point registers share the same physical storage.  The
# `Overlaps' field indicates that the storage of the given register is
# the same than the referred register.  In order to avoid redundancy
# on the database we follow the following rules on expressing overlapping
# relationships, enforced by the constraints below:
#
# - Single-precision FP regs do not overlap.
# - Double-precision FP regs can overlap single-precision FP regs.
# - Quad-precision FP regs can overlap both double-precision and
#   single-precision FP regs.
%type: Overlaps rec Register
%constraint: Type = "FPR" && Width = 32  => !#Overlaps
%constraint: Type = "FPR" && Width = 64  => #Overlaps => Overlaps ~ "^FS[0-9]+"
%constraint: Type = "FPR" && Width = 128 => #Overlaps => Overlaps ~ 
"^F[DS][0-9]+"
# Fields are described by:
#  NAME START-BIT END-BIT PERMS
# where PERMS is one of `r', `w' or `rw'.
%type: Field regexp /^ *[a-z_A-Z0-9]+ +[0-9]+ +[0-9]+ +(r|w|rw) *$/
# The value of some registers are accessed by special instructions.
# The `Getter' and `Setter' fields are used to specify them in the
# records.
%type: Getter rec Instruction
%type: Setter rec Instruction