Re: [pdf-devel] [patch] convert LZW filter to new API

[Juan Pedro Bolivar Puente]

-----BEGIN PGP SIGNED MESSAGE-----
Hash: SHA1

Hi Brad,

> I think the test case is OK. It is directly from the PDF specification, so 
> the 
> test data should be OK. I could, of course, have made an error in the 
> implementation.
> 
> Both decode and encode work with my patch (excluding the buffer size issue, 
> which I'm not sure I fully understand - there is a fixed overhead associated 
> with the initialization and the 9-bit symbols, and you can't avoid the output 
> growing in this case).
> 

You are right, the test cases are ok. I have reviewed the standard and
your patch to find out that there is aproblem in the filter system. Your
patch actually works because it is not considering ENINPUT cases -which
is actually what makes it fail when cache < than input as it assumes EOF.

The problem is that there is a point in the LZW decoding where there is
no way to distinguish between an EOF and ENINPUT. I've reviewed
pdf_stm_filter_apply and what is happening is:

  1. The ambiguous situation where different behaviour for EOF/ENINPUT
is required arises. The LZW decoder returns ENINPUT.
  2. pdf_stm_filter_apply checks the LZW decoder return value. On
ENINPUT it then checks the actual input to see if there is more data to
feed the filter.
  3. As there is no more data, then it asumes EOF, and returns.
  4. As a result, there are pending decoded bytes in the internal state
that are never flushed because the filter did not know that an EOF ocurred.

Jemarch, as you implemented the filter logic, what do you think that is
the best solution for this? I can try to fix it in my patch but as it
affects the internal filter API it probably should be discussed here before.

>> If Brad agrees I can try to fix the cache needs problem and send a patch
>> in few days -this time I won't dissapear, I promise :p.
> I'm more than happy for you to work on this. Feel free to use whatever makes 
> sense (to you) from your current code and mine.
> 

Ok, your code is being of great help, many thanks :)

JP
-----BEGIN PGP SIGNATURE-----
Version: GnuPG v1.4.9 (GNU/Linux)
Comment: Using GnuPG with Mozilla - http://enigmail.mozdev.org

iEYEARECAAYFAkpDmE4ACgkQchi8veCammf+swCfWd+jkMB2yKv1yTFtJxbaSMoj
gMMAnjBSexWLmwy8HtEgZvd3NBv8tZlx
=CNgo
-----END PGP SIGNATURE-----