Re: [lmi] Unknown fields in table input text files

[Vadim Zeitlin]

On Sun, 21 Feb 2016 19:15:43 +0000 Greg Chicares <address@hidden> wrote:

GC> On 2016-02-21 18:12, Vadim Zeitlin wrote:
GC> > 
GC> >  This won't work, actually. The reason is that the order of the tables in
GC> > the original .dat disk file is lost when doing this and I see no possible
GC> > benefit in preserving it (not that it would be possible when doing it 
table
GC> > by table anyhow).
GC> 
GC> Was reordering part of the old '--compress'?

 If you mean "--squeeze", yes, it was.

GC> Even if it wasn't, would it be easier to implement a sort-tables-by-number
GC> option (which would make the shell code above work as desired)

 Such option is easy to implement and even right now we could use --rename
doing nothing to emulate it, but it still wouldn't make the shell code
above work because any database created by our code (and this is the same
for both the legacy and the new versions) will always have the tables in
the same order in the index and in the data file but this is not
necessarily the case for the existing databases, so the files won't be
exactly equal.

GC> To me, making the round trip with every file (including proprietary
GC> tables that you don't have, need, or IMO want) is an obvious and
GC> powerful high-level test that may uncover "unknown unknowns". That's
GC> why I think it's quite important.

 I understand this but I'm just trying to say that the existing test gives
you 99.99% of this. I'll definitely regret having said this when you
demonstrate one to me, but I just can't imagine any input that would pass
the existing test but wouldn't pass the test you'd like to have (notice
that I can imagine plenty of inputs that would not pass the existing test,
of course).

 Anyhow, I'll implement the missing 0.01% tomorrow.

 Regards,
VZ