Re: [Savannah-hackers-public] [gnu.org #337760] [sr #105912] Home page upload broken?

[Sylvain Beucler]

On Mon, Jul 16, 2007 at 03:11:23PM -0400, Justin Baugh via RT wrote:
> > I don't think it is a good idea to move services to Savannah while it
> > is already a loaded computer.
> 
> We must have different definitions for loaded.
> 
> Savannah's load is trivial, considering it's a quad Xeon.
> Over the last year, measured by our cacti instance, the 5 minute 
> load average is < 2.0. Over the last few months, that drops to 
> 0.96. In addition, iostat seems to suggest that the disk
> subsystem is under 10% utilization. And with over 1gb of ram
> inactive it's not pressed for memory. 
> 
> Not only that, but nongnu.org is static web content. It is unlikely
> that the addition of this would have any noticeable effect on 
> Savannah's performance profile at all.
> 
> I don't want to press the issue. If you don't want to do this, or
> don't see the utility in it, it can simply stay where it is now, and
> everyone agrees that the current system needs to be improved.

In the current state of things, I guess this isn't a problem indeed.
We still have some high load peaks though:
http://savannah.gnu.org/cgi-bin/uptimes.cgi
There's probably something I don't understand in analysing load averages..

I don't have much time right now for this; however that's something we
can do without sysadmin intervention until the DNS change, so we'll
contact you when everything is ready on the Savannah side.

> > gnu.org however is much more complicated because every project may be
> > replicated at a different point of the hierarchy (software/, /,
> > translation projects, education/, and other exceptions).
> > gnu.org is probably where improvements are needed the most.
> 
> What improvements, in your mind, should be made? There are a number of 
> potential improvements that come to my mind...

I think we could specify where the replication should take place for a
given project, at initialization time, with the ability to change it
later on.


> As for what has been done, all the scripts involved in the process 
> (cron jobs, new project creation script) now do comprehensive 
> logging and check to make sure that they are not stomping on each
> other (the latter was an improvement made a long while ago). Debugging
> any problems should be much easier. Indeed, several were exclusively
> related to the migration to Apache 2.0, and shouldn't reoccur.
> 
> One thing I noticed is that scripts that are called from Savannah
> return nothing in the way of output. A possibility is to actually 
> return a status code and message - either that, or an XMLRPC interface 
> to replace the given system could be designed, which would actually
> return any useful information including error messages, and also allow
> a variety of useful operations (re-checkout a project from scratch, 
> make a new project, update a project, etc).

They return nothing because at a point we found it easier that way for
including in cron jobs - but I think it was just curl misconfiguration
from our end.

On the other hand, the fact it returns generic 404 whenever there's a
problem is pretty misleading; getting that fixed would be nice.

As far as I'm concerned I don't really care about what technology is
used to trigger to update. However, please keep it simple; I
appreciate being able to trigger the update using a simple
command-line CURL call. Writing XML for testing/fixing is no fun.

> There's no real reason on our end why we can't do on-commit updating
> now. I'm willing to spend the time getting it done, along with any
> other improvements you want made.

Btw, we already do on-commit update, by just calling the python script
again. I just doesn't update at the right place for projects that are
GNU and not mapped under /software.

-- 
Sylvain