Re: [bug #40225] Deterministic output ordering

[Tim Murphy]

>> The idea of diffing 2 builds is truly a cool one - especially when
>> they're huge - but I'd rather it was done according to keys or other
>> factors e.g. target name.
>> I'd like to compare the time it took to build various targets and find
>> out why a target took so much longer in the new build. I'd like to
>> know "what errors did this build gain compared to the last one" or
>> "what errors were fixed?" Another is: what targets didn't get produced
>> that could have been?

>I'm not sure I understand the last point (with pattern rules, there
>are potentially infinitely many target that could have been
>produced, but aren't needed), but otherwise, sure, these are all
>interesting features. However, I do think they fall in different
>groups. The original request was about ordering the existing output
>produced by the commands. What you describe is additional metadata
>that make itself would have to generate.

Targets that would have been built but they're uptodate.

Basically there is this hammer, called diff, and it's attractive to
try to hit everything with it even though it was aimed at files with
relatively few, localised changes - since they are edited by humans.
So here we are thinking about trying to make output look like
something it isn't so that we can get away with using a rather limited
tool on it.

You can knock up a diff tool in quite short order in python using
difflib or using set() intersections and difference and you can base
the comparison on all sorts of things other than contiguous lines of
text. The thing that's missing is the metadata to let you do it.

You can even post-process a log into some stable order and then use
diff if you must - but not without the other information.

This is far from rockets science - it's not even 'interesting' - it's
being actually done by other tools.   Some of these tools produce a
machine readable format which is then filtered to give more human
readable output - users can even choose which filters to apply or they
can apply them after the build is over to a log file and look at it in
different ways.

There's not much make needs to do here - most of this is for other
tools to do - it only has to support a very few little enabling
features and then you can organise the build output any way you like
and I can do it in a way that suits me and, basically, everyone gets
cake.

Regards,

Tim

On 11 October 2013 04:10, Frank Heckenbach <address@hidden> wrote:
> Edward Welbourne wrote:
>
>> Then commit all the log files to git and use git show to find out which
>> of them have changed since the last build.  (I trust you can all work
>> out the equivalent steps for *your* preferred SCM system.)  This only
>> has to happen on the server that builds from clean on a regular basis.
>
> Or just use diff?
>
>> In particular, note that diffing build logs is only useful for a
>> build-from-clean such as a build server does; for incremental builds,
>> deterministic ordering would merely interfere with the immediacy of
>> seeing my stuipid mistakes so that I can start fixing them while the
>> rest of the build runs, ready for the next incremental.
>
> Of course, for similar reasons we now have different levels of
> output-sync, and that's why I think ordering can be seen as yet
> another one of those levels.
>
> Tim Murphy wrote:
>
>> The idea of diffing 2 builds is truly a cool one - especially when
>> they're huge - but I'd rather it was done according to keys or other
>> factors e.g. target name.
>> I'd like to compare the time it took to build various targets and find
>> out why a target took so much longer in the new build. I'd like to
>> know "what errors did this build gain compared to the last one" or
>> "what errors were fixed?" Another is: what targets didn't get produced
>> that could have been?
>
> I'm not sure I understand the last point (with pattern rules, there
> are potentially infinitely many target that could have been
> produced, but aren't needed), but otherwise, sure, these are all
> interesting features. However, I do think they fall in different
> groups. The original request was about ordering the existing output
> produced by the commands. What you describe is additional metadata
> that make itself would have to generate.
>
> Again, I'm not denying the latter could be useful. Paul once
> mentioned the idea of creating a '"machine interface" mode for make,
> as some other GNU CLI tools like GDB, etc. have.' Perhaps some of
> those metadata (run time, targets, dependencies ...) could be
> produced in this mode as well.
>
> On the other hand, ordering commands' output seems quite a different
> thing (at least implementation-wise), and since we've just
> implemented output-sync and still know how it works ;), now might be
> a good time ...
>
> Paul Smith wrote:
>
>> > $(wildcard ...)
>>
>> Anyway, that's up to the makefile author to deal with.
>
> So we agree here.
>
> About the feasibility, of course my suggestion is probably missing
> some details, but it still doesn't seem that difficult to me. Do you
> see any fundamental problem? Of course, normally this would be the
> time for me to provide a reference implementation, but as I said,
> I (or my clients) don't have a really big need for the feature right
> now ...
>
> _______________________________________________
> Bug-make mailing list
> address@hidden
> https://lists.gnu.org/mailman/listinfo/bug-make



-- 
You could help some brave and decent people to have access to
uncensored news by making a donation at:

http://www.thezimbabwean.co.uk/friends/