Re: [Monotone-devel] line endings as project policy

[Richard Levitte - VMS Whacker]

In message <address@hidden> on Wed, 22 Nov 2006 10:05:06 -0500, address@hidden 
said:

hendrik> >  - We need to treat files as binary unless told otherwise.
hendrik> >    This I regard as a fact.  (see the problem with screwed
hendrik> >    up files without the user knowing about it)
hendrik> 
hendrik> Agree.  This is an essential safety constraint.

Good.

hendrik> >  - We need to mark text files as such.  This I regard as
hendrik> >    fact, and it seemt to me like this is almost concensus.
hendrik> 
hendrik> Agree.

Good.

hendrik> >  - We need to convert line endings to the local standard on
hendrik> >    anything that's assumed to be text on checkout.  This I
hendrik> >    regard as a fact.  (see the problem that some Unixly
hendrik> >    programs have with embedded \r)
hendrik> 
hendrik> This seems obvious, but I have some discomfort with the idea.
hendrik> Perhaps because I'm thinking of the wider issues involved in
hendrik> character set incompatibility.  IN any case, conversion on
hendrik> checkout should be overridable in some way.

Aye, I hear ya.  Character set incompatibility (and conversion) is a
bigger can of worms.  I believe we can handle that separately.  We do
need to handle that as well at some point.  But for now, I'd like to
keep it small and manageable and focus on line ends, if for nothing
else then *so we get something done*.

hendrik> If we use an internal line ending standard, we should
hendrik> consider the possibility of using the standard newline
hendrik> character NEL, "Next Line", 0x85, unicode U+0085.

Now, there's an idea...  Just be ready for getting pounced over that
one, that would mean some rather big changes, me thinks.

hendrik> > Let's get it right and reach consensus instead, well
hendrik> > grounded into are minds and our wills.
hendrik> 
hendrik> To get it really well-grounded, we might also consider it in
hendrik> the context of character set conversion.  Points that are
hendrik> easy to overlook with respect to line endings may be
hendrik> glaringly obvious in this larger context.  Even if we don't
hendrik> solve the larger context, it may make decisions clear with
hendrik> the smaller one.

I'm all for hearing about things that will help resolve the smaller
issue first.

hendrik> > So, anything I forgot?
hendrik> 
hendrik> Just how do we mark files as being text in the data base?
hendrik> Will it conceptially be part of the checked-in revision, and
hendrik> editable and mergible like anything else?

I was imagining an attribute.  They can be set, fetched and dropped.
Attributes are conceptually part of the checked-in revision.  I guess
the biggest thing to resolve is what happens if that attribute changes
for some reason, and that includes merging where the atribute value
doesn't match in both parents.

Come to think of he, how does merging of non-matching attributes work
today?  For example, when a file is marked as executable in one
revision but not in the other, and a merge is performed down the line?

hendrik> Just how does the user mark files as being text?  A specific
hendrik> parameter on initial checkin, to be changed later on checkin?
hendrik> A default for new files based on the last few letters of the
hendrik> name?  A sanity check whether the file is really of the type
hendrik> claimed?

Answered in order:  1,2) I propose an attribute, 3) a default of some
sort helps as well, and doesn't stop the user from making changes (njs
proposed .mtn-autoprops, and we already have something that detects
binary...  sorry, manual_merge files), 4) well, as sanity check might
be a good thing, if we think it's needed.  We don't have a sanity
check of that sort today, though...

hendrik> Can we uncompress compressed files so as top better
hendrik> diff/merge the contents and recompress on checkout?  This
hendrik> might be very helpful for openoffice files.

Uhmmmmmm, I seriously thing this is way out of monotone's scope.

hendrik> How do we handle the transition between the current
hendrik> conventions and the new ones?

We currently do not convert by default.  When the lua function
get_linesep_conv() is undefined, monotone works as if it returned
{'LF','LF'}, and when the two elements are exactly the same, monotone
does no conversion whatsoever.  None at all, as far as I remember (I
can't be arsed to look at the code for the moment, so find out on your
own or wait a day or two for me to confirm or take back that statement
:-)).

hendrik> Are we currently storing files as unicode or UTF-8?  (I think
hendrik> only admin information such as file names)  Should we store
hendrik> text files as UTF-8?

Uhmmmm, I'm under the impression that we store as UTF-8.  Damn, it's
been too long since I looked at the code, my DRAM apparently needs a
refresh...

Cheers,
Richard

-----
Please consider sponsoring my work on free software.
See http://www.free.lp.se/sponsoring.html for details.

-- 
Richard Levitte                         address@hidden
                                        http://richard.levitte.org/

"When I became a man I put away childish things, including
 the fear of childishness and the desire to be very grown up."
                                                -- C.S. Lewis