ltib
[Top][All Lists]
Advanced

[Date Prev][Date Next][Thread Prev][Thread Next][Date Index][Thread Index]

Re: [Ltib] Multi-user LTIB


From: Stuart Hughes
Subject: Re: [Ltib] Multi-user LTIB
Date: Wed, 13 Feb 2008 16:11:30 +0000

Hi Ken,

I can't really comment on your requirement without knowing the details
(which would not be appropriate), however we do have an
autobuild/autotest system we use with LTIB at the core.   See inline for
specific responses

On Wed, 2008-02-13 at 10:29 -0500, Ken Gilmer wrote:
> Hi Stuart,
> 
>    I appreciate your detailed response!  It took me some time to go  
> through your email and make sure I understand everything.  I  
> certainly agree in this approach for LTIB in general.  I think what  
> we are trying to do with LTIB may be outside of it's intended use and  
> perhaps not appropriate.  Here are the "requirements" I see that we  
> are looking for:
> 
> 1. Support continuous build and test automation.  Specifically we  
> wish to hook our SCM to CruiseControl and to perform builds and test  
> executions in an entirely automated fashion.
> 2. Allow multiple developers to utilize an existing Linux system  
> concurrently and independently.
> 3. Allow users with a high variance in expertise to use the system.   
> We may want to allow customers to generate their own target images.   
> This may present some kind of opportunity for a web UI on top of LTIB  
> and package metadata.
> 
>    In regards to the points regarding the reasoning behind the  
> existing behavior, I humbly submit counter points:
> 
> > * Ensuring that all content referenced in packages is the same
> > (/opt/{ltib,freescale}/pkgs
> 
> What is the relevant distinction between packages living in /opt/...  
> and in a private package pool?  Disregarding performance for the  
> moment, are there other reasons?

Stuff in /opt/... is not guaranteed to be globally unique (e.g. it
hasn't yet been shared out).  It's not until you upload to some kind of
package pool that at that point you consider a filename taken up.  Of
course there exists the potential for independent uploads of same named
files to different package pools, but this doesn't happen often and if
necessary some arbitration (rename) needs to take place.

At a more basic level, stuff in /opt can only be shared on that machine
whereas in a package pool anyone on your network can get a file and know
its the "official" version of the file with that name.

> 
> >
> > * Reducing overall installation size: if each instance
> > had /opt/{ltib,freescale}/pkgs, it would be a big overhead
> >
> 
> % du --max-depth=1 -h .
> 1.4G    ./com.buglabs.build.production
> 
> % df -h
> Filesystem            Size  Used Avail Use% Mounted on
> /dev/md/2             229G   41G  177G  19% /
> 
> % du --max-depth=1 -h /opt
> 1.7G    /opt/freescale
> 103M    /opt/ltib
> 
> Ok, yes, 3.1 Gb is not small.  But this can be considered temporary  
> (does not grow with each successive build) as we would generate this  
> fresh each time.  In addition discs are big and getting bigger (and  
> cheaper).
> 

What have multiple copies of the same set of files? apart from the waste
of space you end up with 2 users with file x that has the same name and
is a different meaning.  Only when you pull your independent efforts
together do you realise this.  It's even worse than that, even on the
Internet you can find different versions of essentially the same package
(pkg-version.tar.bz2) that have effectively the same content but have
different md5sums.  All the stuff from a package pool has an md5sum
check to make sure you get what you expect.  Also by definition if your
users share /opt on a machine then they get the same content.

BTW: You can of course in addition have per-instance content in
rpm/SOURCES and this will win (be the first path searched).  However the
sooner this gets to /opt or better still a package pool the better.

> 
> > * Sharing the common host support packages so that when something  
> > breaks
> > I can be sure what environment (host tools) this happened (for most
> > things).  The discipline this imposes is that backward  
> > compatibility has
> > to be ensured (which I try hard to do).
> >
> 
> Understood.  From a support perspective, assuming there was a  
> configuration change (like a "mode") to switch, we could isolate  
> behavior by switching back to standard "mode" to isolate potential  
> issues.

Yes, we recently ran into genext2fs which changed the meaning of -i from
inodes to inode ration.   We patch this so that when running from an
LTIB environment it keeps the old meaning (and warns).

> 
> In any case I understand and agree with the reasons LTIB works the  
> way it does.  From my perspective (potential) developer time  
> resolving inconsistencies that come from concurrency issues outweigh  
> the fixed-costs that a purely isolated LTIB system would incur.  I  
> guess my question now:  Is it appropriate to modify LTIB to work the  
> way we want it do?  We certainly do not wish to hack it such that  
> we're running our own custom version.  If the modifications required  
> are much more than pure (global) configuration it probably doesn't  
> make sense.
> 

You need to break down each of your requirements into a simple to
understand rationale and present them.  If they make sense and don't
conflict with some other existing design goal then there's no reason why
we can't modify LTIB to work in that way.  As always "the devils in the
detail" so you need to break it down so I can understand it.

Regards, Stuart









reply via email to

[Prev in Thread] Current Thread [Next in Thread]