Re: [GNU-linux-libre] Re: any Free BSD variant?

[Karl Goetz]

On Thu, 18 Jun 2009 01:48:00 +0300
Waver Doganov <address@hidden> wrote:

> Graziano Sorbaioli wrote:
> > So.. what do you think could be the best system/method to use as a
> > base to create a "GNU os"?
> 
> Short answer: Every GNU-based distro is a variant of the GNU OS.  The
> fact that all of them (except a small bunch which are basically
> maintained as a delta to the master distro, and have no reason to
> exist if the master distro was free) include non-free software is an
> indication that the free software movement is not yet where we want it
> to be: most of the users don't care about these issues, and developers
> often trade them for popularity's sake.

Update: I see while I've had this email open for editing others have
replied. Ah well. Lets see if I manage to confuse the conversation.
 
> Very long answer:

There are times in my reply where I wasn't sure if you were referring to
gNewSense (hereafter gNS) or your ideal distro. As such I might be
talking on a tangent in some replies - apologies in advance.

> I'll try to explain my personal views, hoping this won't be (too)
> annoying for the list readers.  Apologies if it turns out to be
> undesirable noise.

While I'm in disclaimer mode: I am a maintainer of gNS, but what I say
here about the future isn't set in stone (or even reasonably cured
concrete) - some of its 'what I'd like' and not been discussed in a
meaningful way.

> I've been thinking about this for years, starting from my migration
> from Red Hat to Debian long time ago, the surprising discovery that
> Debian turned out to be not so much about freedom despite the slogans,
> the subsequent migration to gNewSense days after its first release and
> my 2+ years old attempt to maintain some packages for gNS at
> http://gns.katsarov.org [1].

Are these packages site specific, or generic?

> The long term solution, in my view, is to develop an absolutely free
> distro from scratch, of course basing it off (initially at least, and
> probably forever for the packaging-specific toolchain) an existing
> one.

No small task.

> Yes, this is unfortunate duplication of effort.  But I see no other
> way.  (This was considered before, multiple times, and rms has
> rejected the idea of a pure "GNU" distro.  Instead, he has suggested
> to raise specific concerns and cooperate actively with communities
> like Debian's.  This is mission impossible, IMNSHO.  They just don't
> listen to us, and there's no way to change that.)

I don't feel 'they just don't listen to us' is an accurate reading of
the situation. Many of them do listen, but will wait for a patch to fix
<the problem>. 

> accumulated through the years, I started to write a document to
> describe them precisely.  Some details are still unclear, and
> naturally I don't expect everyone to automatically grok the concept.
 
> I realize it is difficult to propagate such an idea without a proper
> documentation.  Sadly, I'm far from finishing that document (it is
> larger than the GNU Coding Standards, and close to the size of the
> Debian Policy).  Once completed at least in the form I want it to be,

I'll be interested to see the document - draft or final.

> I'll propose it to the gNewSense maintainers for consideration.  (I
> don't have an ETA for that -- I'm literally drowning in various tasks;
> if I manage to read the thousands mails I receive every day, it's a
> good day...)

Don't worry, we have lots on too :)

> Anyway, here is an incomplete list of features/requirements of my
> dream system just to give you an idea (I point out several
> deficiencies in gNS just for clarity):

Heres where some of my confusion came in. I'll try and comment in two
parts: On the idea, on gNS in our current situation and short-term
situation.

> * The system should never ever contain non-free parts/components, no
>   matter how unpopular the decision about the removal is.  This may
>   seem self-explanatory, but it's not, actually.  This leaves no place
>   for exceptions, like the (in)famous Debian GRs about the firmware in
>   the Linux kernel.  A perfect example of this policy in action was

I agree with the 'no exceptions'
I'm interested to hear how you think this situation could be handled:

 1 week before release 2 bits of non-free firmware are found in the
kernel. Would you delay release? remove them in an update ASAP after
release (to allow testing)? something else?

>   the removal of OpenGL in gNS.  However, it was not properly done:
>   lots of packages were failing to build and/or run as a result of
>   this removal, even though they worked fairly well without GL.  It
>   was not properly done because there was no sane way to do it
>   properly with the way gNS is being developed; and that's the whole
>   point.

Yes, that is partly an unfortunate effect of how Builder operates, and
partly a lack of interest in the community to patch Builder so it did
The Right Thing (eg, rebuild other packages).

gNS 3 will handle package building differently (How exactly I don't want
to go into right now, but basically a cut down autobuilder).

>   more about freedom, so Debian follows them.  However, non-free parts
>   discovered in texlive are not removed in stable and oldstable
>   releases.  My dream distro would treat these as "serious" bugs which
>   warrant an update in stable and oldstable (when supported).

Debians policy is not to change the archive post release - it would be
a matter of policy to add a clause 'freedom bugs mean package removal'
as we do in gNS, and the system under discussion would be good to go.

> * GNU packages by design are intended to work together: this is
>   inherently a very important and core decision, because the ultimate

(trim)

>   will by default use GNU packages that are intended for certain
>   tasks, instead of the (far more popular) non-GNU alternatives.  In
>   no doubt this will help overall with the enhancement of the GNU
>   system.  FWIW, I somewhat partially agree with the widespread
>   opinion that in the recent years lots of packages of dubious quality
>   (like my own :-)) had become GNU packages, so the decision about
>   switching to the GNU alternative should be deeply thought.

Switching to GNU packages is a nice long term goal, but where the system
is the only one using them in a certain way it puts all the burden of
testing and integration onto that distro. Long term, perhaps a good
goal, I won't try and do it in gNS though.
 
>   In short, this means that by default the following will happen:

(trim)

>   This goal is very ambitious, no doubt about that.  But I'm
>   absolutely sure that it would be a good thing for GNU.  Making them
>   default at least will make them more tested, especially in
>   conjunction with the rest of the system.

Agreed.
 
> * The system should not assume that the user is a clueless dummy.

Thank you.

>   According to my interpretation this does not contradict with the
>   goal the system to be easy to use, which is something desirable.
>   The problem here is that "easy to use" != "something that looks like
>   Windoze || MuckOS".  Now, things like command-not-found are nice as
>   a concept, and have noble goals, but how annoying it is to wait

Pet peeve at this end too.

Especially when it winds up with bugs like this: 
 http://bugs.debian.org/cgi-bin/bugreport.cgi?bug=524896
Summary:
FreeDesktop.org mandates a dozen directories in $HOME/ and they will be
recreated if you remove them.

>   nearly a minute on a slow machine thinking that the program does its
>   job, while at the end discovering that the CPU cycles went for
>   Packages.gz parsing?  Users just need to learn which packages
>   provide the programs they need, and packages like command-not-found
>   are not a replacement.

Or need to learn how to search for the command they want (`apt-cache
search coreutils` or synaptic -> find -> coreutils for example).

>   Things like annoying distro logos plastered everywhere and extremely
>   ugly logout dialogs that were rejected upstream *for a reason* come

Sometimes upstreams are difficult (not knowing the dialogue your
talking about I cant comment, but I thought I'd throw that idea into
the mix). Debian bug 531221 could be seen as an example of that.

> * In continuation of the above item, the system should include
>   debugging symbols by default (via Recommends).  Every arch-dep

This I can't agree with. The overhead in cpu/memory/disk/network usage
cant be justified by the few times that debugging symbols are needed.

>   package should have a -dbg companion.  As a user, I want full power,

However I do agree that every package should have a -dbg build
available.

>   and this includes the ability to debug the world.  I want this
>   functionality at the tip of my fingers, although I don't do it too
>   often.  When I do it, I am very annoyed to rebuild packages with
>   DEB_BUILD_OPTIONS=nostrip and/or to modify+build those which do not

Yeah, doing this is annoying.

(trim)

>   There is a reason why the GNU Build System (and most other build
>   systems @rant{which appeared only because people were lazy enough to
>   read the fine GNU manuals}) include debugging symbols by default.

One reason is of course they only build and host a handful of packages,
not 15,000.

>   Naturally, sometimes this is not desirable (small hard disk, etc.),
>   hence the Recommends relationship.

I think a better option would be an option in package management to
enable installing -dbg for each package. It allows opt in if desired,
and doesn't introduce overheads for all those who /dont/ debug.

For example, one could add it like this:
cat <<EOF>> /etc/apt/apt.conf.d/01-enable-debug-packages
APT::INSTALL::DEBUG=yes
EOF
 
> * Documentation.
> 
>   gNS Delta* is derived from Ubuntu, so fortunately includes the free
>   GNU manuals of many essential packages.  Unfortunately, they have
>   humiliating names inherited from Debian, like "texinfo-doc-nonfree"
>   and source package names like "make-dfsg" (which presumes GNU make

I can't find a 'make-dfsg'.

>   is not entirely free).  It is not feasible at all to fix this in gNS
>   because of the way things work right now (with Builder).  Heck, I

Which will change with 3 (I'll let you imagine me saying that whenever
Builder comes up from this point. I'm not saying the problem will
disappear, but the solution will be different, hopefully easier).

>   bet it's not feasible to fix this even in Ubuntu because of the way
>   they derive from Debian.

It is possible, but it requires carrying deltas against the package,
anything that depends on it, and everything that references it. In the
case of texinfo-doc-nonfree its a trivial amount. For other packages
its probably more.

address@hidden:~# apt-cache rdepends texinfo-doc-nonfree
texinfo-doc-nonfree
Reverse Depends:
  texinfo
  info
 
>   A user installing a particular package almost certainly wants the
>   documentation as well, or will need it sometime in the future (when
>   she will eventually fall in the "sysadmin trap" described above).
>   It should be installed by default (either included in the main
>   binary package (or -dev if it's a library and the docs are small) if
>   it is likely to be needed (such as Autoconf, etc.) or in a separate
>   -doc package if it is large).  The main binary package should
>   recommend the -doc package.  (No hard dependency because of the same
>   reasons about the debugging symbols, and because it makes no sense
>   program to Depend: on program-doc apart from sparing some buildd
>   cycles, iff the docs are built by the binary-indep target).

I think documentation of any reasonable size should be in its own
package - when doing embedded systems (or even stripped back systems)
its often desirable to save that 40mb of docs.

>   They are hard to maintain if not done upstream (this is true for any
>   documentation format, of course).  I have personally never submitted
>   upstream even one of those I wrote myself, because they make no
>   sense and cannot be handled by upstream's build system (GNUstep Make
>   in my case).  I absolutely hated every minute spent in nroff-mode
>   (this is subjective, surely).

Theres a number of 2man packages in Debian for this very reason. The
man page format sucks to write.

>   This "should" Debian Policy directive will be replaced by "Every
>   program should contain documentation, in the most appropriate form
>   suitable for it".  A requirement for documentation in Info format is
>   nice, but not realistic to be adopted upstream.  Besides, Texinfo

The nice thing about man is the skill required to use it: none.
Navigating through info is like nethack (only the monsters are scarier
when using info)

> * Every package (ideally) will have a dedicated maintainer, who will
>   ensure that it is always up-to-date and properly maintained, to the
>   best of her ability.  This is the hardest (and most unrealistic)
>   part, but it solves numerous fundamental issues, like:

I dislike the single maintainer concept.

>   - The maintainer will ensure that the package is entirely free,
>     because she will review the diff between every upstream release,
>     thus constantly verifying the package's suitability for inclusion.
>     Efforts like PFV/KFV are admirable, but they'll never be
>     completed, and are a waste.  Understandably, the gNS maintainers

Yes, {P,K}FV are suboptimal.

>     cannot review the diff between two releases, even if they happen
>     to be extremely masochistic.

We're bad, but not that bad ;)
 
>     That way, fewer non-free packages will unintentionally sneak in
>     and they'll be less buggy as the maintainer will not be just a
>     packaging/syncing bot (what we have in gNS now, at least for the
>     packages that are not that popular).

I assume your talking about the 'bootstrapped from the ground' system,
rather then gNS itself. 

>   - The maintainer will update the package in time, making the system

What is "in time"?

>     a viable platform for software development, business and all sorts
>     of common tasks that someone could use a computer for.

This leaves me wondering what your idea of the stable release is -
snapshots like you would find in Gentoo? or a release like those from
Debian?
 
>   - Debian's concept is to have a maintainer (be it a real person or a
>     team) for each package, while Ubuntu's is relaxed: all developers
>     form a giant QA team.  Both policies have their strengths and

Sort of. Ubuntu effectively has two big QA teams (currently). MOTU and
Core-dev. MOTU can touch universe/multiverse, Core-dev can touch
anything.

>     weaknesses -- Debian's favors work by people who know the code and
>     devote themselves to maintaining the package, while Ubuntu's
>     allows almost everyone to touch everything when there's a need.
>     Unfortunately, the former means that sometimes it gets a long time
>     for something (even trivial) to be fixed.  Unfortunately, the

The latter also has this problem.
For example:
 https://bugs.launchpad.net/ubuntu/hardy/+source/trac/+bug/86685
Summary: python-clearsilver is broken on amd64, causing Trac to fail.

>     latter means that people with relevantly little knowledge about
>     the package's code base, programming language, or packaging, can
>     unintentionally introduce lots of bugs.  The dream system will
>     have a policy which is a mixture of both practices -- dedicated
>     maintainers, but very relaxed NMU policy (even for bugs that are
>     not RC), and possibly a review system.

I prefer a groups based model:
Each package is maintained by a group, of which one or two people are
the lead maintainers. This means anyone in the group can upload a fixed
package if needed, but there is a quantity of direct knowledge from the
package leads.

The same group could be responsible for multiple packages (ala Debians
Python/perl/games/gnome/etc teams), where different people lead on
different packages.
 
>   Yes, I realize that finding maintainers for all packages is a HUGE
>   task.  Just like GNU, I believe it _can_ happen.  Some Debian
>   developers/maintainers have similar views, and would be willing to
>   contribute.  Other enthusiasts can learn Debian packaging,
>   eventually.  This is going to be a long process, so we're coming to
>   the next point:

Its a long term goal, definitely.
 
> * The releases will be developed pretty much like Debian.  This
>   ensures a consistent archive and no embarrassing bugs like those in
>   gNS (which is based on a "hyper-stable" Ubuntu LTS release).

Can you expand on "embarrassing bugs"? I'm interested.

>   Basically almost every non-popular package in gNS has at least one
>   important bug, simply because Canonical said they'll release on that
>   date and you can't stabilize an entire distro with magic.

While I dislike Ubuntus release process, I don't agree that "least one
important bug" is an accurate assertion.

>   A list of packages which have a dedicated gNS maintainer will be
>   maintained, and all the others will be semi-manually synced from
>   Debian (testing), into a gNS analogue of sid, leading to a freeze
>   sequence (toolchain, non-essential toolchain, libraries, all),
>   ending up in a supposedly stable release.

I would rather not have a Sid, other then for 'in house' packages. I'd
prefer migrating things in from Debians testing into a gNS testing.
Applying a 'No RC" policy to testing, and aiming for a release every 18
months.

> * Launchpad integration^Wamputation.
> 
>   This goes without saying.  Even when it becomes free, Launchpad will
>   not be entirely free.  All translations should be maintained
>   upstream, period.  As an active translator I should mention that
>   many things are b0rked in gNS, including my own stuff.  Ubuntu, with
>   the noble goal of lowering the contribution barrier, has made
>   something bad: Translators skip the compile-test-compile-test circle
>   which has many nefarious effects.  It is also useless to maintain a
>   separate translation system -- all improvements should go upstream
>   for everybody's benefit.
> 
>   Reporting bugs should always happen through the distro channel,
>   currently consistently achieved in Debian via reportbug, debian-el
>   or reportbug-ng (the latter is somewhat frowned upon, for good
>   reasons).  The "Launchpad integration" stuff just adds an
>   unnecessary diff only for the GUI programs they care about.  My
>   GNUstep packages don't have such a "Translate this program..." or
>   "Report a bug..."  menu.  And widely loved GNU programs don't have
>   either.  Guess why?  Because GNUstep is not Ubuntu's top priority,
>   and they apparently think that a user of a typical GNU package knows
>   enough about how to report bugs.

Only default install/main gets the LP hack treatment. Its a relatively
small subset of packages, and I'm pretty confident they didn't consider
GNUstep users ability to report bugs when integrating LP into
GNOME/KDE. It just didn't factor.

> * Infrastructure.
> 
>   Debian's infrastructure, although not ideal, is entirely free and
>   already proved for the extremely large size of the distribution and
>   the diversity of the developer community.  It is also very
>   convenient for people who often do not work in a X session (like me)
>   or prefer/have to work offline.  Sure, reporting bugs with reportbug
>   rather than the web-based LP may be shocking to some users.  But the
>   ultimate goal should be to teach them how to live and enhance the
>   Free World we all live in, rather than to offer and encourage
>   alternative inconvenient routes they're already familiar with.

Yep. ( Reportbug even had a GTK UI in the past, it may be possible to
dig it up and polish it off.)

>   There is absolutely no point to reinvent the wheel here -- the BTS,
>   PTS, dak, britney and other RT scripts, buildd/sbuild/wanna-build,
>   etc. can be simply reintroduced (with slight modifications which
>   should be eventually submitted to Debian if made wise) for this
>   dream-distro.  This would have the benefit of Debian people
>   contributing with little effort, among other things.

The build system part of this is planned for gNS 3. Builder will stay
around for the life of 2.x, but only maintaining DeltaH. MetaD releases
(those currently based on Debian) will be the test cases, and we'll
hopefully move everything over for 3.


> [1] The server is unreliable, so might be offline.
> 
>     The packages in `jungle' are not Ubuntu-like merges from Debian
>     unstable, they are essential forks.  I decided it's worth to

Forks of Debian packages or Ubuntu packages?

>     diverge because I had to maintain a delta anyway (dpkg in gNS did
>     not support the substvar ${binary:Version}, for example) and I had
>     actually the opportunity to express my views about the dream
>     system in practice.  And I wanted to provide the GNUstep port of
>     Emacs when it was merged in CVS trunk.  And I had to fix bugs I
>     considered worth fixing, etc.  I planned to add more packages to
>     the list, but that's a big burden: getting familiar with the code,
>     subscribing to all upstream mailing lists, keeping up with
>     development, etc. are things that unavoidably slow you down.
> 
>     This repository is strictly unofficial and was not publicized at
>     all, intentionally -- mostly because I'm not certain about my own
>     abilities and I want a general, distro-wide solution, not PPA-like
>     unofficial additions like my own `jungle' creation.

Out of interest, do you check them with lintian? (tool to perform
checks for Debian policy compliance).

> 
> [2] I work for a shipping company as a shipbroker and ship manager.
> 
>     Unfortunately, a migration Debian -> gNS is out of question
>     currently because gNS does not guarantee upgrades (an essential

2->3 will also be unsupported, but hopefully 3->4->onwards will be
supported.

>     feature of every system, I'd say), even though we're otherwise

I agree.

>     ready to do it (only my boss uses 2 non-free packages on his
>     laptop, and I'll happily buy a network card with my own money for
>     one machine where the driver is removed because it has blobs).

kk

-- 
Karl Goetz, (Kamping_Kaiser / VK5FOSS)
Debian contributor / gNewSense Maintainer
http://www.kgoetz.id.au
No, I won't join your social networking group

signature.asc
Description: PGP signature