Re: [Savannah-users] Savannah VMs rebooting for kernel upgrades

[Bob Proulx]

Bob Proulx wrote:
> Question for the mailing list users: Is this the type of information
> that you find useful to be posted to the savannah-users mailing list?

Rather than many individual replies I am going to create a combined
reply here for the several responses.

Gordon Haverland wrote:
> I am not yet doing much in the way of VMs myself, but I have friends
> that ask me about VMs (since I know a lot more about computers in
> general).  Reading things like this (or like this with more detail)
> give me more information as to where to look for answers for my
> friends.
>
> It is likely this is atypical.

Probably there are many people who would like to learn more about VMs
(virtual machines).  But we would probably run off other
savannah-users who would like to keep the list focusted only on those
issues.  Please write me directly offlist about VMs as of a general
nature and we can make further discussion.

Eric Wong wrote:
> Yes.  Any updates like this one about downtime and/or upgrades are
> greatly appreciated.  In my experience, even "transparent" upgrades
> can cause user-visible behavior changes, so I appreciate being
> informed.  Thank you.

I will say "Good!" to this.  Because reboots will kill any actively
running process that is happening at the moment the reboot occurs.
This means that some people cloning a git repository might see an
error at that moment.  Mostly they might just try it again but a
continuous integration system cloning might log failures in that
cycle.  Also someone looking at the web page right at those critical
moments might see a mangled display.  Hopefully they will wait a
moment and reload the page.  That type of thing.

"André Z. D. A." wrote:
> I have no opinion for that question because I do not use some things that
> happens around or that Savannah has. What these machines do?

This could actually be a very long and involved answer.  Which is why
I saved this one for the end of the email.  So that I could leave the
message with a list of things that are affected by a reboot.  But
first let me give a short, extremely abbreviated, summary.  Really!

There is one system that is a dedicated database system.  It is
currently running MySQL.  It holds all of the account information for
all of the subscribers.  Upgrading MySQL or rebooting this system
causes account data to be unavailable during the upgrade and reboot.
Perhaps 1-2 minutes of downtime depending upon the upgrade.  If
account data is not available then no one can authenticate with the
version control system.  If the database is not available then the web
site is not fully functional.

There is the main web server frontend.  This is one of three web
servers in the collection.  It hosts what most people probably
visualize when they think of Savannah.  But really the web interface
is only a control panel to add and remove people and features from
projects.  All of the real work is done elsewhere.  If the web server
is rebooting and unavailable then no one can add or remove features or
people from projects.  It is dependent upon the database server for
both web data and account data.  No one can post new project news
items.  If the web server is unavailable no one can look at the
documentation there.  Perhaps 1-2 minutes of downtime while the system
reboots.

There is a version control system which hosts all of the various
version control systems.  All of git, svn, cvs, bzr, hg, run on this
system.  If the version control system is rebooted then all active
processes are killed.  Someone checking out a repository, or a
continuous integration system doing so, will have a failure when their
running process is killed.  During the time the system is rebooting
none of the version control systems are available.  Connection
attempts will fail with connection errors until it arrives back
online.  Additionally the version control server is dependent upon the
database server for account information.  Perhaps 1-2 minutes of
downtime while the system reboots.

There is a project file download server.  Project files are stored
there and made available for download.  The database server must be
online in order for account information to be available to
authenticate project members to be able to upload files there.
Perhaps 1-2 minutes of downtime while the system reboots.

Additionally there are two legacy data file system servers.  These are
network file systems mounted upon the systems described above.  They
are operated on a different upgrade cycle and are not rebooted as
often.  They are firewalled off from the hostile Internet.  As
currently provisioned when those are rebooted it may require file
system checks.  They have a large amount of data.  If so then the
checks may run for an hour to complete before the file servers are
back online.  This is an area of ongoing improvement moving forward.

That is the abbreviated summary of the system.  Really that is
abbreviated!  There are actually many more details that I didn't even
hint at or "gloss over". :-)

Bob

The following is a long list of public services that are associated
with Savannah and all have at least a momentary disruption when the
collection of servers are rebooted or individual services upgraded.
Yes there is a lot of similar but different URLs for exactly the same
service.  I don't know why.  It's all historical legacy.

http://savannah.gnu.org/
http://savannah.nongnu.org/
https://savannah.gnu.org/
https://savannah.nongnu.org/

bzr branch bzr://bzr.savannah.gnu.org/
bzr branch bzr://bzr.savannah.nongnu.org/
bzr branch bzr://bzr.sv.gnu.org/
bzr branch bzr://bzr.sv.nongnu.org/
bzr branch http://bzr.savannah.gnu.org/lh/
bzr branch http://bzr.savannah.gnu.org/r/
bzr branch http://bzr.savannah.nongnu.org/lh/
bzr branch http://bzr.savannah.nongnu.org/r/
bzr branch http://bzr.sv.gnu.org/lh/
bzr branch http://bzr.sv.gnu.org/r/
bzr branch http://bzr.sv.nongnu.org/lh/
bzr branch http://bzr.sv.nongnu.org/r/
bzr branch https://bzr.savannah.gnu.org/lh/
bzr branch https://bzr.savannah.gnu.org/r/
bzr branch https://bzr.savannah.nongnu.org/lh/
bzr branch https://bzr.savannah.nongnu.org/r/
bzr branch https://bzr.sv.gnu.org/lh/
bzr branch https://bzr.sv.gnu.org/r/
bzr branch https://bzr.sv.nongnu.org/lh/
bzr branch https://bzr.sv.nongnu.org/r/
bzr branch bzr+ssh://address@hidden/
bzr branch bzr+ssh://address@hidden/
bzr branch bzr+ssh://address@hidden/
bzr branch bzr+ssh://address@hidden/

rsync://bzr.savannah.gnu.org/bzr/
rsync://bzr.savannah.nongnu.org/bzr/
rsync://bzr.sv.gnu.org/bzr/
rsync://bzr.sv.nongnu.org/bzr/

http://bzr.savannah.gnu.org/lh
http://bzr.savannah.gnu.org/lh/
http://bzr.savannah.gnu.org/r
http://bzr.savannah.gnu.org/r/

cvs checkout -d :ext:address@hidden:/cvsroot/
cvs checkout -d :ext:address@hidden:/sources/
cvs checkout -d :ext:address@hidden:/srv/cvs/sources/
cvs checkout -d :ext:address@hidden:/srv/cvs/web/
cvs checkout -d :ext:address@hidden:/web/
cvs checkout -d :ext:address@hidden:/webcvs/
cvs checkout -d :ext:address@hidden:/cvsroot/
cvs checkout -d :ext:address@hidden:/sources/
cvs checkout -d :ext:address@hidden:/srv/cvs/sources/
cvs checkout -d :ext:address@hidden:/srv/cvs/web/
cvs checkout -d :ext:address@hidden:/web/
cvs checkout -d :ext:address@hidden:/webcvs/
cvs checkout -d :ext:address@hidden:/cvsroot/
cvs checkout -d :ext:address@hidden:/sources/
cvs checkout -d :ext:address@hidden:/srv/cvs/sources/
cvs checkout -d :ext:address@hidden:/srv/cvs/web/
cvs checkout -d :ext:address@hidden:/web/
cvs checkout -d :ext:address@hidden:/webcvs/
cvs checkout -d :ext:address@hidden:/cvsroot/
cvs checkout -d :ext:address@hidden:/sources/
cvs checkout -d :ext:address@hidden:/srv/cvs/sources/
cvs checkout -d :ext:address@hidden:/srv/cvs/web/
cvs checkout -d :ext:address@hidden:/web/
cvs checkout -d :ext:address@hidden:/webcvs/
cvs checkout -d :pserver:address@hidden:/cvsroot/
cvs checkout -d :pserver:address@hidden:/sources/
cvs checkout -d :pserver:address@hidden:/srv/cvs/sources/
cvs checkout -d :pserver:address@hidden:/srv/cvs/web/
cvs checkout -d :pserver:address@hidden:/web/
cvs checkout -d :pserver:address@hidden:/webcvs/
cvs checkout -d :pserver:address@hidden:/cvsroot/
cvs checkout -d :pserver:address@hidden:/sources/
cvs checkout -d :pserver:address@hidden:/srv/cvs/sources/
cvs checkout -d :pserver:address@hidden:/srv/cvs/web/
cvs checkout -d :pserver:address@hidden:/web/
cvs checkout -d :pserver:address@hidden:/webcvs/
cvs checkout -d :pserver:address@hidden:/cvsroot/
cvs checkout -d :pserver:address@hidden:/sources/
cvs checkout -d :pserver:address@hidden:/srv/cvs/sources/
cvs checkout -d :pserver:address@hidden:/srv/cvs/web/
cvs checkout -d :pserver:address@hidden:/web/
cvs checkout -d :pserver:address@hidden:/webcvs/
cvs checkout -d :pserver:address@hidden:/cvsroot/
cvs checkout -d :pserver:address@hidden:/sources/
cvs checkout -d :pserver:address@hidden:/srv/cvs/sources/
cvs checkout -d :pserver:address@hidden:/srv/cvs/web/
cvs checkout -d :pserver:address@hidden:/web/
cvs checkout -d :pserver:address@hidden:/webcvs/

rsync://cvs.savannah.gnu.org/sources/
rsync://cvs.savannah.nongnu.org/sources/
rsync://cvs.sv.gnu.org/sources/
rsync://cvs.sv.nongnu.org/sources/

http://cvs.savannah.gnu.org/
http://cvs.savannah.gnu.org/viewcvs/
http://cvs.savannah.gnu.org/viewvc/
http://cvs.savannah.nongnu.org/
http://cvs.savannah.nongnu.org/viewcvs/
http://cvs.savannah.nongnu.org/viewvc/
http://cvs.sv.gnu.org/
http://cvs.sv.gnu.org/viewcvs/
http://cvs.sv.gnu.org/viewvc/
http://cvs.sv.nongnu.org/
http://cvs.sv.nongnu.org/viewcvs/
http://cvs.sv.nongnu.org/viewvc/
http://web.cvs.savannah.gnu.org/viewcvs/www/
http://web.cvs.savannah.gnu.org/viewvc/www/
http://web.cvs.savannah.nongnu.org/viewcvs/www/
http://web.cvs.savannah.nongnu.org/viewvc/www/
https://cvs.savannah.gnu.org/
https://cvs.savannah.gnu.org/viewcvs
https://cvs.savannah.gnu.org/viewcvs/
https://cvs.savannah.gnu.org/viewvc
https://cvs.savannah.gnu.org/viewvc/
https://cvs.savannah.nongnu.org/
https://cvs.savannah.nongnu.org/viewcvs/
https://cvs.savannah.nongnu.org/viewvc/
https://cvs.sv.gnu.org/
https://cvs.sv.gnu.org/viewcvs/
https://cvs.sv.gnu.org/viewvc/
https://cvs.sv.nongnu.org/
https://cvs.sv.nongnu.org/viewcvs/
https://cvs.sv.nongnu.org/viewvc/
https://web.cvs.savannah.gnu.org/viewcvs
https://web.cvs.savannah.gnu.org/viewcvs/www
https://web.cvs.savannah.gnu.org/viewcvs/www/
https://web.cvs.savannah.gnu.org/viewvc
https://web.cvs.savannah.gnu.org/viewvc/www
https://web.cvs.savannah.gnu.org/viewvc/www/
https://web.cvs.savannah.nongnu.org/viewcvs/www/
https://web.cvs.savannah.nongnu.org/viewvc/www/

http://download.savannah.gnu.org/releases/
http://download.savannah.nongnu.org/releases/
http://download.sv.gnu.org/releases/
http://download.sv.nongnu.org/releases/
https://download.savannah.gnu.org/releases/
https://download.savannah.nongnu.org/releases/
https://download.sv.gnu.org/releases/
https://download.sv.nongnu.org/releases/

http://download-mirror.savannah.gnu.org/releases/
http://download-mirror.savannah.nongnu.org/releases/
http://download-mirror.sv.gnu.org/releases/
http://download-mirror.sv.nongnu.org/releases/
https://download-mirror.savannah.gnu.org/releases/
https://download-mirror.savannah.nongnu.org/releases/
https://download-mirror.sv.gnu.org/releases/
https://download-mirror.sv.nongnu.org/releases/

rsync://download.savannah.gnu.org/releases/
rsync://download.savannah.nongnu.org/releases/
rsync://download.sv.gnu.org/releases/
rsync://download.sv.nongnu.org/releases/

git clone git://git.savannah.gnu.org/
git clone git://git.savannah.nongnu.org/
git clone git://git.sv.gnu.org/
git clone git://git.sv.nongnu.org/
git clone http://git.savannah.gnu.org/git/
git clone http://git.savannah.gnu.org/r/
git clone http://git.savannah.nongnu.org/git/
git clone http://git.savannah.nongnu.org/r/
git clone https://git.savannah.gnu.org/git/
git clone https://git.savannah.gnu.org/r/
git clone https://git.savannah.nongnu.org/git/
git clone https://git.savannah.nongnu.org/r/
ssh://address@hidden/srv/git/
ssh://address@hidden/srv/git/
ssh://address@hidden/srv/git/
ssh://address@hidden/srv/git/
rsync://svn.savannah.gnu.org/git/
rsync://svn.savannah.nongnu.org/git/
rsync://svn.sv.gnu.org/git/
rsync://svn.sv.nongnu.org/git/

http://git.savannah.gnu.org/
http://git.savannah.gnu.org/cgit/
http://git.savannah.gnu.org/git/
http://git.savannah.gnu.org/gitweb/
http://git.savannah.gnu.org/r/
http://git.savannah.nongnu.org/
http://git.savannah.nongnu.org/cgit/
http://git.savannah.nongnu.org/git/
http://git.savannah.nongnu.org/gitweb/
http://git.savannah.nongnu.org/r/
http://git.sv.gnu.org/
http://git.sv.gnu.org/cgit/
http://git.sv.gnu.org/git/
http://git.sv.gnu.org/gitweb/
http://git.sv.gnu.org/r/
http://git.sv.nongnu.org/
http://git.sv.nongnu.org/cgit/
http://git.sv.nongnu.org/git/
http://git.sv.nongnu.org/gitweb/
http://git.sv.nongnu.org/r/
https://git.savannah.gnu.org/
https://git.savannah.gnu.org/cgit/
https://git.savannah.gnu.org/git/
https://git.savannah.gnu.org/gitweb/
https://git.savannah.gnu.org/r/
https://git.savannah.nongnu.org/
https://git.savannah.nongnu.org/cgit/
https://git.savannah.nongnu.org/git/
https://git.savannah.nongnu.org/gitweb/
https://git.savannah.nongnu.org/r/
https://git.sv.gnu.org/
https://git.sv.gnu.org/cgit/
https://git.sv.gnu.org/git/
https://git.sv.gnu.org/gitweb/
https://git.sv.gnu.org/r/
https://git.sv.nongnu.org/
https://git.sv.nongnu.org/cgit/
https://git.sv.nongnu.org/git/
https://git.sv.nongnu.org/gitweb/
https://git.sv.nongnu.org/r/

hg clone http://hg.savannah.gnu.org/hgweb/
hg clone http://hg.savannah.nongnu.org/hgweb/
hg clone http://hg.sv.gnu.org/hgweb/
hg clone http://hg.sv.nongnu.org/hgweb/
hg clone https://hg.savannah.gnu.org/hgweb/
hg clone https://hg.savannah.nongnu.org/hgweb/
hg clone https://hg.sv.gnu.org/hgweb/
hg clone https://hg.sv.nongnu.org/hgweb/
hg clone ssh://address@hidden/
hg clone ssh://address@hidden/
hg clone ssh://address@hidden/
hg clone ssh://address@hidden/

http://hg.savannah.gnu.org/hgweb/
http://hg.savannah.nongnu.org/hgweb/
http://hg.sv.gnu.org/hgweb/
http://hg.sv.nongnu.org/hgweb/
https://hg.savannah.gnu.org/hgweb
https://hg.savannah.gnu.org/hgweb/
https://hg.savannah.nongnu.org/hgweb
https://hg.savannah.nongnu.org/hgweb/
https://hg.sv.gnu.org/hgweb/
https://hg.sv.nongnu.org/hgweb/

rsync://hg.savannah.gnu.org/hg/
rsync://hg.savannah.nongnu.org/hg/
rsync://hg.sv.gnu.org/hg/
rsync://hg.sv.nongnu.org/hg/

svn checkout http://svn.savannah.gnu.org/svn/
svn checkout http://svn.savannah.nongnu.org/svn/
svn checkout http://svn.sv.gnu.org/svn/
svn checkout http://svn.sv.nongnu.org/svn/
svn checkout https://svn.savannah.gnu.org/svn/
svn checkout https://svn.savannah.nongnu.org/svn/
svn checkout https://svn.sv.gnu.org/svn/
svn checkout https://svn.sv.nongnu.org/svn/
svn checkout svn+ssh://address@hidden/
svn checkout svn+ssh://address@hidden/
svn checkout svn+ssh://address@hidden/
svn checkout svn+ssh://address@hidden/
svn checkout svn://svn.savannah.gnu.org/
svn checkout svn://svn.savannah.nongnu.org/
svn checkout svn://svn.sv.gnu.org/
svn checkout svn://svn.sv.nongnu.org/

rsync://svn.savannah.gnu.org/svn/
rsync://svn.savannah.nongnu.org/svn/
rsync://svn.sv.gnu.org/svn/
rsync://svn.sv.nongnu.org/svn/

http://svn.savannah.gnu.org/
http://svn.savannah.gnu.org/viewcvs/
http://svn.savannah.gnu.org/viewvc/
http://svn.savannah.nongnu.org/
http://svn.savannah.nongnu.org/viewcvs/
http://svn.savannah.nongnu.org/viewvc/
https://svn.savannah.gnu.org/
https://svn.savannah.gnu.org/viewcvs
https://svn.savannah.gnu.org/viewcvs/
https://svn.savannah.gnu.org/viewvc
https://svn.savannah.gnu.org/viewvc/
https://svn.savannah.nongnu.org/
https://svn.savannah.nongnu.org/viewcvs/
https://svn.savannah.nongnu.org/viewvc/
http://svn.sv.gnu.org/
http://svn.sv.gnu.org/viewcvs/
http://svn.sv.gnu.org/viewvc/
http://svn.sv.nongnu.org/
http://svn.sv.nongnu.org/viewcvs/
http://svn.sv.nongnu.org/viewvc/
https://svn.sv.gnu.org/
https://svn.sv.gnu.org/viewcvs
https://svn.sv.gnu.org/viewcvs/
https://svn.sv.gnu.org/viewvc
https://svn.sv.gnu.org/viewvc/
https://svn.sv.nongnu.org/
https://svn.sv.nongnu.org/viewcvs/
https://svn.sv.nongnu.org/viewvc/