Re: [Savannah-hackers-public] Anonymous HTTPS cloning of emacs git repository is failed

[Bob Proulx]

Yasuhiro KIMURA wrote:
> I'm reporting here because I was recommended in emacs-devel ML.

Yes.  This is the right place for the report.  Thanks for reporting it.

> For about half a day anonymous HTTPS cloning of emacs git repository
> has been failing as following.
> 
> yasu@eastasia[3561]% git clone https://git.savannah.gnu.org/git/emacs.git
> Cloning into 'emacs'...
> error: RPC failed; HTTP 504 curl 22 The requested URL returned error: 504
> fatal: the remote end hung up unexpectedly

It's a strange problem.  I am not immediately sure why this is failing.  It
had been working.  It works for other projects.  It works for git-daemon
protocol.  It works for ssh member protocol.  It works for the raw dumb http
protocol.  It is only the git-http-backend that this times out after exactly 1
minute.

Therefore this feels like something specific to the emacs.git repository such
as something that has been corrupted recently.  However once again the other
protocols succeed. Using a fast local system a full emacs clone takes 6
minutes.  Remote clones over the Internet can only take longer.

The 1 minute timeout on the surface seems like the default proxy_read_timeout
60s nginx setting.  However for the other protocols interaction begins within
2 seconds.  Also changing that timeout to 180s had no change and the http 504
timeout still occurred at exactly 1 minute. Therefore this seems to be a
different timeout.

Just by way of a workaround you might use the raw repository dumb http
protocol for access.  It is slower but it should work.  Note also that
the dumb http protocol gives little feedback until the very end when
suddenly it is reported as finished.  So for progress reporting it is
needed to do du and ls listings in a different terminal in order to
know that it is doing anything.

    rwp@mgt0:~/src/emacs-stuff$ time git clone 
https://git.savannah.gnu.org/r/emacs.git
    Cloning into 'emacs'...
    Checking connectivity... done.
    Checking out files: 100% (4126/4126), done.

    real    9m24.158s
    user    6m0.027s
    sys     1m9.163s

However obviously the git:// protocol is faster and has better
feedback.  The raw repository dumb http protocol does have the
advantage of using https however.

Further debugging...  All of the files are readable.  I don't see any
obvious errors.  I made a copy for safety and I ran a git fsck on a
copy of the repository and had this result.

    root@vcs0:~/rwp/emacs.git# git fsck
    Checking object directories: 100% (256/256), done.
    Checking objects: 100% (901650/901650), done.
    Checking connectivity: 903357, done.
    dangling commit 53d8419fb4d30e4537f714e2bc0facf2d93693cd
    dangling commit 324683df526cfdceffde5936e27b074f4a091208
    dangling commit 16bd454fa837f50fd5f1d159e8f152c597592415
    dangling commit 0b3a68ed521bd526714fb261126805c94d30b828
    dangling commit 56a64ec49a4693e965da158b8fa5a9858b9abf88
    dangling commit 4409ef6deade193058ad74f80c41eb72265645d0
    dangling commit d067d05eb06d613f5b1be3f558eb88c814eda9fc
    dangling commit fb4c51743bcf33efad8718b0e8cd54dfadc0a6ae
    dangling commit 3ae232d923608ca3561385dc587e619e0ac9fcc5
    dangling commit 612df640cbcc800c14768f0722e5cd7001faa5f0
    dangling commit 8e3df6bc291616858f7858d283a1b2da610398a2
    dangling commit 0382368015bfcdb7b88dd68a739f2bc5cde77bce
    dangling commit 0f28d7ae1c4a28a35a3ae7047e4c246cc2e1c846
    dangling commit ad4879d9717cc116008f4083573bdbcceb782dd6
    dangling commit d8e6b9ba8af329e009a2d675c43ba678412690df
    dangling commit 666bda627c265cec7c2ab4de6bfa4b9a6c1c0568
    dangling commit 4b8edaf3046d6ddfa50a7965dfab7ed6baaacf90
    dangling commit ae921bb85a7a452554e72551783e859780b27e11
    dangling commit 8d085d2f8d3a9deb82cdfd319a4206abb043951e
    dangling commit 555efd91c9815849f241afbf97ea32ed8c577c1f

I'll go ahead and send this update and keep looking into why this is failing.

Bob