Re: Malformed sav file - need help for investigation

[Ben Pfaff]

Ah. I can easily explain what the problem is.

SPSS data files have a variable record for each variable. For string
variables with a width greater than 8, they actually have a variable record
for every 8 columns of width (rounding up). Lots of places in data files
refer to variables by the index that includes such extra records. So, if
you have a string variable with width 24, and it's the third variable in
the file following two numeric variables, then it will have indexes 2, 3,
and 4. When such a variable is referenced by index, only index 2 is
supposed to be used. PSPP complains if index 3 or 4 is used instead, and
SPSS (apparently) crashes instead.

Maybe I should make PSPP more tolerant, just giving a warning instead of an
error. If so, then it would be easy to fix up a file with this error by
using pspp-convert.

On Wed, Jun 10, 2020 at 10:24 AM Ignácz, István <istvan.ignacz@neticle.com>
wrote:

> Thanks, Ben for the quick response!
>
> Yes, the ultimate goal is to add a patch to the package which fixes the
> problem or throws an error if the user of the package (in this case me)
> provides invalid input. I think I'm facing the later.
>
> One significant difference between the test file and my case is that in my
> case no variables and no cases are recognized so the reading process
> basically crashes.
> I think I've found the part where the PSPP ends up in a 'goto error'
> (sys-file-reader.813) but I lack the context of what's going on here. I
> guess the value labels could be misconfigured? Maybe there's a value in a
> record that doesn't have a label in the variable settings?
>
> If I would know a little bit more about what happens in that part and why
> that causes an error, maybe I could force my (and tiamo's) code to reliably
> produce the error.
>
> Thanks,
> István
>
> On Wed, 10 Jun 2020 at 18:51, Ben Pfaff <blp@cs.stanford.edu> wrote:
>
> > I'm not sure exactly what you're looking for, but PSPP has a test case
> > that intentionally provokes this error message. I'm attaching the file it
> > produces.
> >
> > I guess the ultimate goal here should be to fix tiamo's package?
> >
> > On Wed, Jun 10, 2020 at 9:42 AM Ignácz, István <
> istvan.ignacz@neticle.com>
> > wrote:
> >
> >> Greetings,
> >>
> >> I think this is not the usual topic on this list, excuse me if you find
> it
> >> off-topic.
> >>
> >> I'm facing a tough bug in my software and I'm looking for some help to
> be
> >> able to take the next steps.
> >>
> >> Our software generates sav files via tiamo's package:
> >> https://packagist.org/packages/tiamo/spss
> >> Usually, everything works fine, but in some cases, we generate malformed
> >> files. In this case, SPSS crashes without any notice, however, PSPP at
> >> least gives a useful error message:
> >> Variable index n refers to long string continuation.
> >>
> >> I cannot reproduce this issue on my side, so I tried to reverse engineer
> >> the PSPP codebase to find any cases when it could occur. So far not with
> >> much luck.
> >>
> >> Could you help me with some example cases?
> >>
> >> Thanks for your help,
> >> István
> >>
> >
>
> --
>
> *István Ignácz*
> Software Engineer @ Neticle
>
> +36 30 460 5356
> istvan.ignacz@neticle.com
> neticle.com
>
> <https://neticle.com/mediaintelligence/en/>  <https://zurvey.io/>
> <https://api.neticle.com/>  <https://data.neticle.com/documentation>
>
> Neticle Labs Ltd.
> Address: H-1082 Budapest, Leonardo da Vinci  41. (2nd floor/12)
> Billing address: H-1213 Budapest, Veto street 10.
>