[Top][All Lists]
[Date Prev][Date Next][Thread Prev][Thread Next][Date Index][Thread Index]
Re: patch to support output synchronization under -j
From: |
David Boyce |
Subject: |
Re: patch to support output synchronization under -j |
Date: |
Fri, 15 Apr 2011 10:37:13 -0400 |
On Fri, Apr 15, 2011 at 9:48 AM, Eli Zaretskii <address@hidden> wrote:
> So effectively, whenever a job finishes, its parent Make takes the
> semaphore, outputs all of the output of that job to the screen, then
> releases the semaphore, is that right?
Yes, exactly.
> And how do you communicate the name of the semaphore file to
> sub-Make's? Is that by assuming that its file descriptor is
> inherited?
>
> I also don't understand the need in the MAKESYNCFILE variable: why is
> it important where the semaphore file will live and how will it be
> named?
Addressed by second iteration. There is no sync file anymore.
> Finally, wouldn't it be a potential problem top inherit so many
> handles to subordinate processes (2 for each running job)? We could
> run out of available handles in deeply recursive jobs, couldn't we?
Just to be completely clear, we're not talking about a change to
default behavior - it's merely a new option. But yes, any feature
which consumes additional resources increases the risk of running out
of that resource. I don't know that recursion comes into it though,
it's more a question of how many parallel jobs a given make process is
managing since limits on file handles/descriptors are per process. In
other words if using the jobserver you're limited to the number of
jobs specified at the top level even in a recursive setup. If not,
make tries to work around that by forcing -j1, and if that doesn't
work you're in trouble anyway.
The strictest file limit I know of is on old Unixy platforms. The
tmpfile() API returns a FILE *, not a file descriptor, and older
Unices have a limit of 256 of these. But even in that corner case, how
many people are going to run make -j200+? How many 64-CPU processors
are running really old kernels? So yes, while there is a theoretical
limit there are many kinds of resource exhaustion which can be
triggered by large values of -j and as far as I can tell this is just
another, and optional.
Parenthetically, I only use FILE * because that's what tmpfile hands
back. I believe it would be possible to dup the file descriptor out of
the FILE *, close it right away, and work with an integral file
descriptor from there. That would avoid the above limit.
David
- Re: patch to support output synchronization under -j, (continued)
- Re: patch to support output synchronization under -j, David Boyce, 2011/04/14
- Re: patch to support output synchronization under -j, Tim Murphy, 2011/04/14
- Re: patch to support output synchronization under -j, Eli Zaretskii, 2011/04/14
- Re: patch to support output synchronization under -j, David Boyce, 2011/04/14
- Re: patch to support output synchronization under -j, Eli Zaretskii, 2011/04/15
- Re: patch to support output synchronization under -j, Tim Murphy, 2011/04/15
- Re: patch to support output synchronization under -j, Eli Zaretskii, 2011/04/15
- Re: patch to support output synchronization under -j, Stefano Lattarini, 2011/04/15
- Re: patch to support output synchronization under -j, David Boyce, 2011/04/15
- Re: patch to support output synchronization under -j, Eli Zaretskii, 2011/04/15
- Re: patch to support output synchronization under -j,
David Boyce <=
- Re: patch to support output synchronization under -j, Eli Zaretskii, 2011/04/15
- Re: patch to support output synchronization under -j, Paul Smith, 2011/04/15
- Re: patch to support output synchronization under -j, David Boyce, 2011/04/15
- Re: patch to support output synchronization under -j, Eli Zaretskii, 2011/04/15
- Re: patch to support output synchronization under -j, Tim Murphy, 2011/04/15
- Re: patch to support output synchronization under -j, Paul Smith, 2011/04/15
- Re: patch to support output synchronization under -j, Eli Zaretskii, 2011/04/15
- Re: patch to support output synchronization under -j, Edward Welbourne, 2011/04/19
- Re: patch to support output synchronization under -j, David Boyce, 2011/04/19
- Re: patch to support output synchronization under -j, Howard Chu, 2011/04/15