Re: [Duplicity-talk] large upload "ETA stalled"?

[Cláudio Gil]

I don't have a strong feeling regarding a mark/tag but it does not seem very useful. We could as well add this rough progress reporting to the main backend.py logic and ensure all current and future backends have it. Each backend would then only need to add intermediary progress if it's possible. For backends like boto or dpbx the extra cost would be, more or less, the cost of an extra os.path.getsize(...).

Cheers,
Cláudio

Em 16/01/2017 10:00, <address@hidden> escreveu:

i wonder if it'd make sense to somehow mark/tag backends that support progress and throw an error when --progress is enabled on a backend that currently does not support it?

local file progress should probably properly implement via
  http://bazaar.launchpad.net/~duplicity-team/duplicity/0.8-series/view/head:/duplicity/path.py#L622
to avoid "stalling" on local targets. users might be confused when their otherwise outperforming system claims to be "stalling".

..ede/duply.net

On 16.01.2017 03:08, Cláudio Gil wrote:
> The previous branch now includes the "autoprogress" decorator and uses it
> in the b2 backend. I've also added it to the local backend.
>
> I've done a few tests with the local backend and it works as expected.
> Since it only provides progress at the end of the upload/download then it
> still shows stalled when volumes take longer than 5s (by default) to
> transfer. Using "--progress-rate=<sec>" can avoid that. For example, with
> the default volume size of 200MB and --progress-rate=60 you will see
> "Stalled", one in a while, when the bandwidth is bellow 1.7MB/s (bandwidth
> = volsize/(rate*2)).
>
> Cheers,
> Cláudio
>
> 2017-01-15 12:46 GMT+00:00 <address@hidden>:
>
>> hey Claudio,
>>
>> if it's really that easy you might consider writing a decorator that can
>> be added easily to the remaining backends, that do not need special
>> handling.
>>
>> ..ede/duply.net
>>
>> On 15.01.2017 13:41, Cláudio Gil wrote:
>>> The boto API supports a progress callback which means you can get
>> progress updates during the upload. Dropbox API also supports
>> chunks/appends but I don't think other APIs do. I haven't really looked
>> into it.
>>>
>>> In any case, you are right, the code only introduces progress report for
>> the upload part. It can be easily added to the get (and other backend)
>> because it is a very rough progress report which don't requires any special
>> support.
>>>
>>> Cheers,
>>> Cláudio
>>>
>>> edgar.soldin--- via Duplicity-talk <address@hidden <mailto:
>> address@hidden>> escreveu em dom, 15/01/2017 às 11:30 :
>>>
>>>     On 15.01.2017 12:15, Cláudio Gil via Duplicity-talk wrote:
>>>
>>>     > Hi,
>>>
>>>     >
>>>
>>>     > https://code.launchpad.net/~m4ktub/duplicity/0.7-fixes/+
>> merge/314778 <https://code.launchpad.net/%7Em4ktub/duplicity/0.7-fixes/+
>> merge/314778> <https://code.launchpad.net/%7Em4ktub/duplicity/0.7-fixes/+
>> merge/314778>
>>>
>>>     > If you look at the diff you can see that I've split the import
>> into two. The rest remains the same.
>>>
>>>     >
>>>
>>>     > This kind of progress reporting does not require any support from
>> the backend's API so it would work for all other backends, I suppose. I
>> don't use progress but if others feel important I can try to add this kind
>> of rough progress into the other backends. Maybe this would be 0.8-series
>> material...
>>>
>>>     >
>>>
>>>
>>>
>>>     the progress reporting in _boto_{multi,single}.py looks a bit more
>> complex than that.
>>>
>>>
>>>
>>>     is it possible that this fix only handles the progress for uploads
>> but ignores downloads _get() ?
>>>
>>>
>>>
>>>     ..ede/duply.net <http://duply.net>
>>>
>>>
>>>
>>>     _______________________________________________
>>>
>>>     Duplicity-talk mailing list
>>>
>>>     address@hidden <mailto:address@hiddenorg>
>>>
>>>     https://lists.nongnu.org/mailman/listinfo/duplicity-talk
>>>
>>
>