Re: [Savannah-hackers-public] git and https URLs not working

[Bob Proulx]

Hi Arnold,

Arnold Robbins wrote:
> I'm having trouble pulling from Savannah:
> 
> === groff
> fatal: unable to access 'https://git.savannah.gnu.org/git/groff.git/': Failed 
> to connect to git.savannah.gnu.org port 443: Connection timed out

Hmm...  This is working for me at this time.  However several things
are happening that will cause general instability as a general
statement and some specific things happen that will always make
https/http less reliable.  More in a moment...

> === sed
> warning: expected SRV RR, found RR type 1
> fatal: unable to connect to git.savannah.gnu.org:
> git.savannah.gnu.org[0: 11.11.11.11]: errno=Connection timed out
> git.savannah.gnu.org[1: 11.11.11.11]: errno=Connection timed out

I am not familiar with this error but it feels like a DNS error.  Is
11.11.11.11 your DNS server?  Searching for the message, expected SRV
RR, found RR type 1, seems to be associated with the git:// protocol.
Were you using git:// for that action?

Having said that I think that is definitely still related to the
network changes happening.  You almost certainly saw a network glitch
at that time.  Even though DNS when configured properly with
redundancy is resilient against single failure glitches.

> But I'm able to push to my repo using an ssh URL.

ssh will always be the most reliable protocol method.  For a few
different reasons.

For one fail2ban is tuned very well for ssh attacks.  Attacks against
ssh have been mitigated the most easily.  Therefore the sshd is more
often able to survive attacks.  (Which I hate to say or someone will
purposefully attack it.  We don't have any magic to survive a DDOS.
With today's DDOS attacks if someone wants any site down they just
throw data at it and it remains down until they stop.  Nothing we can
do about it.)

Since ssh access is only for authenticated users the set of
authenticated users is smaller and reduces the effort of the ssh side
of the system to handle git requests.

On the other hand https/http are common targets of attack.  They are
handled by Nginx/Apache which proxy them to the git-daemon which is
the git smart backend service.  The interaction is more complex
because each have a maximum number of connections and they must work
together.  Because https/http also handle other actions and services
it is more difficult to mitigate attacks against them.  It would be
much easier if we had a dedicated server for each version control
service.  Therefore we often get attacks against the https/http port
and it takes everything associated with web servers down until the
attack stops.

Therefore https/http can never be as reliable as ssh.

> Is this related to the IP provider change I saw an email about a week
> or so ago?

Probably.  Since there has been many changes at the Boston datacenter
in order to support the network changes.  I have seen some glitches.
But mostly when I have spot checked I have had connectivity.  While
writing this I ran the full regression test suite and everything passed.

Nothing has yet changed with the public facing Savannah systems
themselves yet.  The change window has now opened to allow us to make
changes but the FSF admins are out of the office until January 2nd for
the Christmas break.

(Well...  They said they were going to be out of the office.  But
being the dedicated individuals that some of them are they have
actually been working this task anyway!  I have been getting emails
about various things related to the change.  But I am going to try not
not to rely upon that and try to avoid making extra work for them.)

Therefore while I changed one of our victim test systems over
I wasn't going to make any changes to the public facing Savannah
systems until they are available to rescue the system in case of
problem.  Because we the Savannah Hacker team do not have console
access.  If anything breaks we must wait until rescue.  (In real life
I am all about the self rescue.  Therefore needing others is always
difficult for me.  But sometimes we all must learn to wait.)

> Thanks,
> 
> Arnold
> (Heading to bed, will see replies in the morning my time.)

Hoping you had a good sleep, with pleasant dreams, are now fully
rested, and feeling ready to take on the world now.  :-)

Bob