Re: [Duplicity-talk] Restart ssh if session not active

[Florian Messmer]

FYI I've looked a bit more at duplicity & paramiko's code.

The two errors ("SSH session not active" and "Socket is closed") seem to be effectively linked. Paramiko creates a new ssh session only when calling `connect()`, but duplicity's `retry()` function only retries the current method (ie put).

What would you think about adding a `_retry_cleanup()` function in ssh_paramiko_backend, that would simply call `self.client.connect()` (and `self.client.open_sftp()` if sftp)? That should prevent any similar problem.

If that sounds ok I could propose a PR, although I don't know if the overhead would be important. Anyway that's only when catching an exception so I guess it's not a big deal, but maybe we should open a new session only for some exceptions?

Regards

Le ven. 8 sept. 2023 à 20:15, Florian Messmer <florian.messmer.1@gmail.com> a écrit :

Thanks for the advice! I actually did read the part about "why use sftp instead of scp" but it didn't really convince me at first glance. :')

Apparently in my case a great side effect is that it's much faster than scp!

Although I still encounter the problem using `sftp`, with a slightly different error message "OSError: Socket is closed" (at least I think, I tried to simulate my network issues by manually restarting the ssh tunnel).

I've then tested with `pexpect+sftp`, I first thought this wasn't working at all, but after investigation only the download rate & progress aren't working (and that's probably normal).

So I enabled sftp logs and did my testing again (restarting ssh tunnel), and it seems to work !!

If I understand correctly the problem is that paramiko doesn't start a whole new session when the socket is closed, while when using `pexpect` the whole sftp command is re-run with auth (and I'd expect the same behaviour with `pexept+scp`).

That would be nice to have this behaviour included with paramiko too (progress bar is quite useful when backing up can take a few days), but I'm not sure if that should be done in duplicity's implementation or in paramiko directly? I found this related issue https://github.com/paramiko/paramiko/issues/998 but the conclusions aren't very promising...

Le ven. 8 sept. 2023 à 11:16, edgar.soldin--- via Duplicity-talk <duplicity-talk@nongnu.org> a écrit :

hey Florian,

why don't you use 'sftp://'? https://duplicity.us/stable/duplicity.1.html#why-use-sftp-instead-of-scp

if your problems persists you may always switch to the legacy pexpect backend which utilizes the command line ssh binaries as a workaround. wrt. the error 'BackendException: scp execution failed: SSH session not active' i have no idea off the top of my hat.

good luck.. ede/duply.net

On 08.09.2023 10:21, Florian Messmer via Duplicity-talk wrote:
> Hi,
>
> I'm using duplicity with scp (on an ssh tunnel), and sometimes when there is an issue with the connection the scp timeout, and I'd like to automatically reconnect if so.
> I configured `backend-retry-delay` and `num-retries` but I seem to always get `BackendException: scp execution failed: SSH session not active` afterwards (even when the client is back online). I'm not an ssh/scp expert but it looks like the details about the previous ssh session are kept thus it can't restore it.
> Any way to force duplicity to "create a new ssh session" after failing?
>
> Regards
> --
> Florian Messmer
>
> _______________________________________________
> Duplicity-talk mailing list
> Duplicity-talk@nongnu.org
> https://lists.nongnu.org/mailman/listinfo/duplicity-talk


_______________________________________________
Duplicity-talk mailing list
Duplicity-talk@nongnu.org
https://lists.nongnu.org/mailman/listinfo/duplicity-talk

--

Florian Messmer

--

Florian Messmer