Re: [Duplicity-talk] Duplicity 0.6.16 "File XXX was corrupted during upload." to s3+http://

[edgar . soldin]

On 03.11.2011 02:18, Michael Terry wrote:
> On 2 November 2011 19:34, Kris <address@hidden> wrote:
>> I'm taking a look at it again right now.  Just a FYI before I dig in, I 
>> removed the volsize cmd line and it did the same thing. The default volsize 
>> is 25MB in duplicity and that's what the tmp file is. The remote file 
>> however is 50 megs. The PoolWorkers split the chunk in to two 25 meg chunks 
>> and uploaded those, which isn't proper.
> 
> Good debugging work, Kris.  This isn't bug 498933 after all, but is
> instead a bug in 0.6.16 specifically.  In that version, the S3 backend
> got new 'multichunk' support.

Well done Kris indeed
 
> But it seems to have a bug that when the size of the volume isn't
> perfectly divisible by the multichunk size, it will upload too much
> data.  I've filed bug https://bugs.launchpad.net/duplicity/+bug/885513
> about it.

Hopefully this can be closed with the patch provided

> 
> For now, I believe you can workaround it with
> --s3-multipart-chunk-size=1000 (causing duplicity to basically ignore
> the multipart code).

Nice, you already added it to the bug 

> 
> I feel like suggesting that maybe the multipart branch should be
> backed out until it can be stress tested a bit more?  (so far we've
> noticed this bug and bug 878220 (the one where botobackend.py added a
> new, uncommon, and unconditional import due to the patch)

How else do you think people would test the new code? If you really feel so, 
maybe you should disable multichunking by default leaving it up to the 
adventurous to try it out.

..ede/duply.net