[GNU-linux-libre] Re: any Free BSD variant?

[Waver Doganov]

Graziano Sorbaioli wrote:
> So.. what do you think could be the best system/method to use as a
> base to create a "GNU os"?

Short answer: Every GNU-based distro is a variant of the GNU OS.  The
fact that all of them (except a small bunch which are basically
maintained as a delta to the master distro, and have no reason to
exist if the master distro was free) include non-free software is an
indication that the free software movement is not yet where we want it
to be: most of the users don't care about these issues, and developers
often trade them for popularity's sake.

Very long answer:

I'll try to explain my personal views, hoping this won't be (too)
annoying for the list readers.  Apologies if it turns out to be
undesirable noise.

I've been thinking about this for years, starting from my migration
from Red Hat to Debian long time ago, the surprising discovery that
Debian turned out to be not so much about freedom despite the slogans,
the subsequent migration to gNewSense days after its first release and
my 2+ years old attempt to maintain some packages for gNS at
http://gns.katsarov.org [1].

My belief is that the task of cleaning up non-free parts from
distributions is humiliating.  Sure thing, far from all free software
developers care about freedom, so repackaging upstream tarballs is
something that's going to be necessary for quite a long time,
unfortunately.  But when you have to clone an entire distro only
because they intentionally include non-free components, this is
insulting.  There should never ever be a need for this exercise,
because distros that have *declared* (e.g. Debian, Fedora) they care
about freedom issues should be entirely non-free-clean by default.
And by an unwritten definition, a GNU distro should be an assembled
GNU system which users can install and upgrade (taking into account
dependencies) without hassle, as opposed to doing it yourself from
scratch.

Thus, I consider the work done by the gNS developers (just an example,
things are probably very similar for other entirely free distros) as
something necessary and very appreciated, but neither viable nor
desirable on the long term.

The long term solution, in my view, is to develop an absolutely free
distro from scratch, of course basing it off (initially at least, and
probably forever for the packaging-specific toolchain) an existing
one.

Yes, this is unfortunate duplication of effort.  But I see no other
way.  (This was considered before, multiple times, and rms has
rejected the idea of a pure "GNU" distro.  Instead, he has suggested
to raise specific concerns and cooperate actively with communities
like Debian's.  This is mission impossible, IMNSHO.  They just don't
listen to us, and there's no way to change that.)

Building something entirely from scratch is another option, but IMHO
worth considering _only_ if a better packaging format than the
currently existing ones is invented.  Of course, people will always
have preferences for a particular system design (Debian/Ubuntu,
Fedora/RHEGL/CentOS, Gentoo, Arch, PLD, to name a few) for various
reasons, so I find it natural free software supporters to unite around
a specific free distro: Debian fans will choose gNS, Gentoo fans will
choose Ututo, Fedora fans BLAG, etc.

My plan is about gNS, because that's what I'm familiar with.  Some
time ago I shared some of my thoughts with Karl Berry (and IIRC, John
Sullivan) from the GNU Project, and he found such an effort
interesting.  However, because many issues and details have
accumulated through the years, I started to write a document to
describe them precisely.  Some details are still unclear, and
naturally I don't expect everyone to automatically grok the concept.

I realize it is difficult to propagate such an idea without a proper
documentation.  Sadly, I'm far from finishing that document (it is
larger than the GNU Coding Standards, and close to the size of the
Debian Policy).  Once completed at least in the form I want it to be,
I'll propose it to the gNewSense maintainers for consideration.  (I
don't have an ETA for that -- I'm literally drowning in various tasks;
if I manage to read the thousands mails I receive every day, it's a
good day...)

Anyway, here is an incomplete list of features/requirements of my
dream system just to give you an idea (I point out several
deficiencies in gNS just for clarity):

* The system should never ever contain non-free parts/components, no
  matter how unpopular the decision about the removal is.  This may
  seem self-explanatory, but it's not, actually.  This leaves no place
  for exceptions, like the (in)famous Debian GRs about the firmware in
  the Linux kernel.  A perfect example of this policy in action was
  the removal of OpenGL in gNS.  However, it was not properly done:
  lots of packages were failing to build and/or run as a result of
  this removal, even though they worked fairly well without GL.  It
  was not properly done because there was no sane way to do it
  properly with the way gNS is being developed; and that's the whole
  point.

  Another example is Tex Live, which is known to contain non-free
  stuff.  This is for historical reasons, and it's gradually removed
  upstream as they're doing audits.  They're removed from Debian as
  well (even simultaneously), but this was not the case with its
  predecessor, TeTeX.  It just happens that TeX Live's upstream cares
  more about freedom, so Debian follows them.  However, non-free parts
  discovered in texlive are not removed in stable and oldstable
  releases.  My dream distro would treat these as "serious" bugs which
  warrant an update in stable and oldstable (when supported).

* GNU packages by design are intended to work together: this is
  inherently a very important and core decision, because the ultimate
  goal was to *replace entirely* the proprietary Unix system, and in
  fact all other non-free systems.  The goal is exactly the same
  today, years after the inception of the project.

  In practice, though, this does not happen (or at least since some
  years).  For example, TRAMP does not work with GNU lsh, GNU spool
  does not work with all the world that wants to print, Guile is not
  yet fully integrated everywhere as initially intended (needless to
  say that Python/Perl/Ruby are more popular), etc.  My dream system
  will by default use GNU packages that are intended for certain
  tasks, instead of the (far more popular) non-GNU alternatives.  In
  no doubt this will help overall with the enhancement of the GNU
  system.  FWIW, I somewhat partially agree with the widespread
  opinion that in the recent years lots of packages of dubious quality
  (like my own :-)) had become GNU packages, so the decision about
  switching to the GNU alternative should be deeply thought.

  In short, this means that by default the following will happen:
  - OpenSSH -> GNU lsh
  - BSD mailx -> GNU mailutils
  - Vixie's cron -> GNU mcron (yes, this means guile is Essential)
  - BSD mktemp -> GNU coreutils (already done in Debian sid) 
  - CUPS -> GNU spool
  - Firefox -> Epiphany or whatever is the default for each desktop
    (done in gNS) and of course Firefox -> GNU IceCat for Firefox itself
  - Apache -> MyServer (why not?)
  - various net-related packages -> GNU inetutils
  ...

  This goal is very ambitious, no doubt about that.  But I'm
  absolutely sure that it would be a good thing for GNU.  Making them
  default at least will make them more tested, especially in
  conjunction with the rest of the system.

* The system should not assume that the user is a clueless dummy.
  According to my interpretation this does not contradict with the
  goal the system to be easy to use, which is something desirable.
  The problem here is that "easy to use" != "something that looks like
  Windoze || MuckOS".  Now, things like command-not-found are nice as
  a concept, and have noble goals, but how annoying it is to wait
  nearly a minute on a slow machine thinking that the program does its
  job, while at the end discovering that the CPU cycles went for
  Packages.gz parsing?  Users just need to learn which packages
  provide the programs they need, and packages like command-not-found
  are not a replacement.

  Things like annoying distro logos plastered everywhere and extremely
  ugly logout dialogs that were rejected upstream *for a reason* come
  to mind, too.  The only reason they exist is because the user-base
  of the distro we're deriving from is used to that kind of look.  Ah,
  and probably because other distros have been doing these silly
  things.  There are thousands of packages in a free system, including
  about 6+ desktop environments and 50+ window managers.  Although
  GNOME in gNS is heavily modified, it is still GNOME.  Distro
  branding must be a limited set of things done wisely and
  consistently.

* In continuation of the above item, the system should include
  debugging symbols by default (via Recommends).  Every arch-dep
  package should have a -dbg companion.  As a user, I want full power,
  and this includes the ability to debug the world.  I want this
  functionality at the tip of my fingers, although I don't do it too
  often.  When I do it, I am very annoyed to rebuild packages with
  DEB_BUILD_OPTIONS=nostrip and/or to modify+build those which do not
  support this option at all and/or to install -dbg packages.
  Especially annoyed when there are shared library calls which are
  important, and I have to do some trivial work to discover what I'm
  missing.  With a slow network connection and aptitude taking ages to
  perform a single command, this is a cold shower.

  Contrary to popular belief, most of the GNU users are not sysadmins
  of the systems they work on.  This is the case at the place I work
  [2] and also at home where people would understand why I can't make
  my (still relevantly young) kids admins of the machines they use.
  Thus, users in need to debug something have to nag the sysadmins,
  and the request is often not fulfilled in time (at least not in time
  until the debugging mood is over).  Another very annoying practice
  is when users send GDB backtraces generated from stripped
  binaries/libraries just because they don't have the -dbg packages
  installed simply because they're not installed by default or do not
  exist at all.  Useless noise and waste of developers' time, which
  could be avoided.

  There is a reason why the GNU Build System (and most other build
  systems @rant{which appeared only because people were lazy enough to
  read the fine GNU manuals}) include debugging symbols by default.
  Naturally, sometimes this is not desirable (small hard disk, etc.),
  hence the Recommends relationship.

* Documentation.

  gNS Delta* is derived from Ubuntu, so fortunately includes the free
  GNU manuals of many essential packages.  Unfortunately, they have
  humiliating names inherited from Debian, like "texinfo-doc-nonfree"
  and source package names like "make-dfsg" (which presumes GNU make
  is not entirely free).  It is not feasible at all to fix this in gNS
  because of the way things work right now (with Builder).  Heck, I
  bet it's not feasible to fix this even in Ubuntu because of the way
  they derive from Debian.

  A user installing a particular package almost certainly wants the
  documentation as well, or will need it sometime in the future (when
  she will eventually fall in the "sysadmin trap" described above).
  It should be installed by default (either included in the main
  binary package (or -dev if it's a library and the docs are small) if
  it is likely to be needed (such as Autoconf, etc.) or in a separate
  -doc package if it is large).  The main binary package should
  recommend the -doc package.  (No hard dependency because of the same
  reasons about the debugging symbols, and because it makes no sense
  program to Depend: on program-doc apart from sparing some buildd
  cycles, iff the docs are built by the binary-indep target).

  It is basically a good thing that Debian requires documentation for
  all programs in /usr/bin and /usr/sbin, but unfortunately as
  manpages.  Manpages are deprecated in GNU and do not really make
  sense for many non-trivial programs.  Lots of manpages in Debian are
  completely useless, made only to satisfy the policy requirement.
  They are hard to maintain if not done upstream (this is true for any
  documentation format, of course).  I have personally never submitted
  upstream even one of those I wrote myself, because they make no
  sense and cannot be handled by upstream's build system (GNUstep Make
  in my case).  I absolutely hated every minute spent in nroff-mode
  (this is subjective, surely).

  This "should" Debian Policy directive will be replaced by "Every
  program should contain documentation, in the most appropriate form
  suitable for it".  A requirement for documentation in Info format is
  nice, but not realistic to be adopted upstream.  Besides, Texinfo
  has certain known severe limitations which I hope will be solved in
  the near future.  Until then (and most probably forever), I can only
  dream of C-h i m <package> working for every package I have
  installed.

* Every package (ideally) will have a dedicated maintainer, who will
  ensure that it is always up-to-date and properly maintained, to the
  best of her ability.  This is the hardest (and most unrealistic)
  part, but it solves numerous fundamental issues, like:

  - The maintainer will ensure that the package is entirely free,
    because she will review the diff between every upstream release,
    thus constantly verifying the package's suitability for inclusion.
    Efforts like PFV/KFV are admirable, but they'll never be
    completed, and are a waste.  Understandably, the gNS maintainers
    cannot review the diff between two releases, even if they happen
    to be extremely masochistic.

    That way, fewer non-free packages will unintentionally sneak in
    and they'll be less buggy as the maintainer will not be just a
    packaging/syncing bot (what we have in gNS now, at least for the
    packages that are not that popular).

  - The maintainer will perform all necessary steps to ensure that the
    package meets this distro's requirements, and not those of
    upstream's.

  - The maintainer will update the package in time, making the system
    a viable platform for software development, business and all sorts
    of common tasks that someone could use a computer for.

  - Debian's concept is to have a maintainer (be it a real person or a
    team) for each package, while Ubuntu's is relaxed: all developers
    form a giant QA team.  Both policies have their strengths and
    weaknesses -- Debian's favors work by people who know the code and
    devote themselves to maintaining the package, while Ubuntu's
    allows almost everyone to touch everything when there's a need.
    Unfortunately, the former means that sometimes it gets a long time
    for something (even trivial) to be fixed.  Unfortunately, the
    latter means that people with relevantly little knowledge about
    the package's code base, programming language, or packaging, can
    unintentionally introduce lots of bugs.  The dream system will
    have a policy which is a mixture of both practices -- dedicated
    maintainers, but very relaxed NMU policy (even for bugs that are
    not RC), and possibly a review system.

  Yes, I realize that finding maintainers for all packages is a HUGE
  task.  Just like GNU, I believe it _can_ happen.  Some Debian
  developers/maintainers have similar views, and would be willing to
  contribute.  Other enthusiasts can learn Debian packaging,
  eventually.  This is going to be a long process, so we're coming to
  the next point:

* The releases will be developed pretty much like Debian.  This
  ensures a consistent archive and no embarrassing bugs like those in
  gNS (which is based on a "hyper-stable" Ubuntu LTS release).
  Basically almost every non-popular package in gNS has at least one
  important bug, simply because Canonical said they'll release on that
  date and you can't stabilize an entire distro with magic.

  A list of packages which have a dedicated gNS maintainer will be
  maintained, and all the others will be semi-manually synced from
  Debian (testing), into a gNS analogue of sid, leading to a freeze
  sequence (toolchain, non-essential toolchain, libraries, all),
  ending up in a supposedly stable release.

* Launchpad integration^Wamputation.

  This goes without saying.  Even when it becomes free, Launchpad will
  not be entirely free.  All translations should be maintained
  upstream, period.  As an active translator I should mention that
  many things are b0rked in gNS, including my own stuff.  Ubuntu, with
  the noble goal of lowering the contribution barrier, has made
  something bad: Translators skip the compile-test-compile-test circle
  which has many nefarious effects.  It is also useless to maintain a
  separate translation system -- all improvements should go upstream
  for everybody's benefit.

  Reporting bugs should always happen through the distro channel,
  currently consistently achieved in Debian via reportbug, debian-el
  or reportbug-ng (the latter is somewhat frowned upon, for good
  reasons).  The "Launchpad integration" stuff just adds an
  unnecessary diff only for the GUI programs they care about.  My
  GNUstep packages don't have such a "Translate this program..." or
  "Report a bug..."  menu.  And widely loved GNU programs don't have
  either.  Guess why?  Because GNUstep is not Ubuntu's top priority,
  and they apparently think that a user of a typical GNU package knows
  enough about how to report bugs.

  This item naturally leads to the next point.

* Infrastructure.

  Debian's infrastructure, although not ideal, is entirely free and
  already proved for the extremely large size of the distribution and
  the diversity of the developer community.  It is also very
  convenient for people who often do not work in a X session (like me)
  or prefer/have to work offline.  Sure, reporting bugs with reportbug
  rather than the web-based LP may be shocking to some users.  But the
  ultimate goal should be to teach them how to live and enhance the
  Free World we all live in, rather than to offer and encourage
  alternative inconvenient routes they're already familiar with.

  There is absolutely no point to reinvent the wheel here -- the BTS,
  PTS, dak, britney and other RT scripts, buildd/sbuild/wanna-build,
  etc. can be simply reintroduced (with slight modifications which
  should be eventually submitted to Debian if made wise) for this
  dream-distro.  This would have the benefit of Debian people
  contributing with little effort, among other things.

Reminding you that this is an incomplete list.  To sum it up, I would
say: Only with dedicated package maintenance and a well-defined
process we can ensure that our distro is/will remain free, and only
this way we can make the system we want it to be.  Certainly, "what we
want it to be" varies when it comes to the technical implementation,
but it is (presumably) the same when we mean ethical considerations.

[1] The server is unreliable, so might be offline.

    This effort started somewhat naturally.  When I migrated my main
    workstation from Debian to gNS, I sensed a big disturbance in my
    workflow, mostly because Ubuntu is a slightly different system
    from Debian (very different, actually).  I did not "suffer" from
    the non-free-removal, because I was not using any non-free
    software anyway, besides the blobs in Debian's kernel.  One of the
    biggest regressions from a usability POV was Emacs 21, while I've
    been accustomed to the not-yet-released 22 via the emacs-snapshot
    package in Debian.  I couldn't even perform my daily tasks without
    referring to the manual for many things; it was that bad.

    So I compiled Emacs from CVS locally.  I kept updating it for some
    time, when I noticed that my brother and a friend were doing the
    same (grumbling because of that).  My workstation was Pentium II,
    and my brother's -- Pentium III.  No fun building Emacs there, it
    takes more than 4-5 hours which is a time I want to spend for real
    work.  So I offered them to provide a debian package, and they
    were very happy.  So it began...

    ...but at some time I needed to regenerate `configure' and Emacs
    had AC_PREREQ(2.61) in its configure.in.  The gNS version of
    autoconf was older.  OK, I tried to build a newer autoconf locally
    but it failed because it needed a newer m4.  I built a modern m4
    as well, but when this was supposed to end?  As a package
    maintainer I can't introduce modifications which cannot be
    reproduced with the versions of the tools available in the distro.
    So I started maintaining m4, then autoconf, then some other
    packages that I considered I'm (close to) capable of maintaining,
    or other people have asked me to.

    Later, a friend offered to host a mailing list for announcements
    of new uploads and discussions, and another friend offered to host
    the repository.

    The packages in `jungle' are not Ubuntu-like merges from Debian
    unstable, they are essential forks.  I decided it's worth to
    diverge because I had to maintain a delta anyway (dpkg in gNS did
    not support the substvar ${binary:Version}, for example) and I had
    actually the opportunity to express my views about the dream
    system in practice.  And I wanted to provide the GNUstep port of
    Emacs when it was merged in CVS trunk.  And I had to fix bugs I
    considered worth fixing, etc.  I planned to add more packages to
    the list, but that's a big burden: getting familiar with the code,
    subscribing to all upstream mailing lists, keeping up with
    development, etc. are things that unavoidably slow you down.

    This repository is strictly unofficial and was not publicized at
    all, intentionally -- mostly because I'm not certain about my own
    abilities and I want a general, distro-wide solution, not PPA-like
    unofficial additions like my own `jungle' creation.

[2] I work for a shipping company as a shipbroker and ship manager.

    We migrated absolutely all machines from Windows to Debian
    GNU/Linux more than 5 years ago (nearly at the time Woody was
    released), and none of the users of the workstations has sysadmin
    rights.  I think this is a sane policy, having in mind their
    knowledge about the system in general.

    Unfortunately, a migration Debian -> gNS is out of question
    currently because gNS does not guarantee upgrades (an essential
    feature of every system, I'd say), even though we're otherwise
    ready to do it (only my boss uses 2 non-free packages on his
    laptop, and I'll happily buy a network card with my own money for
    one machine where the driver is removed because it has blobs).