RE: Recursive implicit rules without explicit intermediates?

[Luke Goodsell]

In case anyone else has a similar task in the future, the interim solution I've 
chosen is to use Make functions to compute the targets explicitly and secondary 
expansion to define prerequisites. This is not without it's limitations - 
notably the awkwardness of overriding targets in specific cases - but it does 
the job relatively easily.

Pasted below is a modified version of my original demonstration Makefile.

Kind regards,
Luke


.SECONDEXPANSION:

GROUPS = group1 group2
SECTIONS = section1 section2

ALL_SECTIONS = \
        $(strip \
                $(patsubst \
                        section%, \
                        section/section%/, \
                        $(SECTIONS)\
                )\
        )

ALL_SECTIONS_ALL_GROUPS = \
        $(strip \
                $(foreach \
                        group, \
                        $(GROUPS), \
                        $(addsuffix \
                                $(group)/,$(ALL_SECTIONS)\
                        ) \
                ) \
        )

dirname = $(patsubst %/,%,$(dir $1))
dirparentdir = $(dir $(call dirname,$1))

all: \
        section/section2/section_end \
        
        echo "$@";

section/section1/section_start: \
        
        echo "$@";

section/section2/section_start: \
        section/section1/section_end
        
        echo "$@";

# eg: section/section1/group1/group_start
$(addsuffix group1/group_start,$(ALL_SECTIONS)): \
        $$(call dirparentdir,$$@)section_start \
        
        echo "$@";

# eg: section/section1/group2/group_start
$(addsuffix group2/group_start,$(ALL_SECTIONS)): \
        $$(call dirparentdir,$$@)group1/group_end \
        
        echo "$@";

# eg: section/section1/group1/step1/do_step
# eg: section/section1/group2/step1/do_step
$(addsuffix step1/do_step,$(ALL_SECTIONS_ALL_GROUPS)): \
        $$(call dirparentdir,$$@)group_start \
        
        echo "$@";

# eg: section/section1/group1/step2/do_step
$(addsuffix step2/do_step,$(ALL_SECTIONS_ALL_GROUPS)): \
        $$(call dirparentdir,$$@)step1/do_step \
        
        echo "$@";

# eg: section/section1/group1/step3/do_step
$(addsuffix step3/do_step,$(ALL_SECTIONS_ALL_GROUPS)): \
        $$(call dirparentdir,$$@)step2/do_step \
        
        echo "$@";

# eg: section/section1/group1/group_end
$(addsuffix group_end,$(ALL_SECTIONS_ALL_GROUPS)): \
        $$(dir $$@)step3/do_step \
        
        echo "$@";

# eg: section/section1/section_end
$(addsuffix section_end,$(ALL_SECTIONS)): \
        $$(dir $$@)group2/group_end \

        echo "$@";




-----Original Message-----
From: address@hidden [mailto:address@hidden On Behalf Of Luke Goodsell
Sent: 16 January 2015 10:47
To: Nicholas Clark; address@hidden
Subject: RE: Recursive implicit rules without explicit intermediates?

Hi Nicholas,

Hi Nicholas,

I think this is probably the most straightforward solution. I'll give this a 
shot.

Many thanks,
Luke

-----Original Message-----
From: address@hidden [mailto:address@hidden On Behalf Of Nicholas Clark
Sent: 15 January 2015 18:18
To: address@hidden
Subject: Re: Recursive implicit rules without explicit intermediates?

How about using pattern substitutions?

Something like:

#-------------------------------------------------------------------#

default: all

#--------- Inputs ------------#

INPUTS := in1.input in2.input in3.input
SECOND_INPUTS := second_in1.input second_in2.input second_in3.input

$(INPUTS) $(SECOND_INPUTS):
    touch $@

#------ Intermediates --------#

INTERMEDIATES := $(patsubst %.input,%.intermediate,$(INPUTS))

%.intermediate: %.input
    cp $< $@

#--------- Outputs -----------#

OUTPUTS := $(patsubst %.input,%.output,$(SECOND_INPUTS))

all: $(OUTPUTS)

%.output: %.input $(INTERMEDIATES)
    cp $< $@

#---------- Misc ------------#

clean:
    rm -f *.output *.intermediate

distclean:
    rm -f *.input *.output *.intermediate

.DELETE_ON_ERROR:
.INTERMEDIATE: $(INTERMEDIATES)




On Thu, Jan 15, 2015 at 4:15 AM, Luke Goodsell <address@hidden>
wrote:

> Hi,
>
> Thanks in advance for taking the time to read and reply to my email.
>
> $ make --version
> GNU Make 4.0
> Built for x86_64-pc-linux-gnu
>
> Abstract:
>
> I'm writing a template makefile for performing similar - but different
> - computational biology pipelines and am encountering an issue with 
> recursive implicit rules. Please can you suggest a solution that 
> doesn't require explicitly listing every intermediate target?
>
> Background:
>
> I'm writing a Makefile template that can be used for new projects. 
> Each project follows a very similar, path but with differences that 
> must be manually adapted. Each project has to perform similar tasks on 
> each of number of different 'sections', each of which has a number of 
> different 'groups' that each have a number of 'steps'. Each step 
> cannot be run until the previous step has run. The same is true of groups and 
> sections.
>
> Problem:
>
> The first group of the first section runs fine, with all steps running 
> without error. However, the second group of the first section - and 
> all subsequent targets - fails with:
>
> make: *** No rule to make target '[TARGET]'.  Stop.
>
> I understand that an implicit rule cannot be used multiple times 
> within a chain, and that the standard solution is to explicitly 
> declare an intermediate. However, since this makefile template may 
> have tens each of steps, groups and sections, that would require 
> hundreds to thousands of explicitly declared intermediates.
>
> Is there a better solution I've overlooked?
>
> Addendum:
>
> A prototypical makefile that demonstrates the problem is pasted below.
>
> Kind regards,
> Luke
>
>
> # Example makefile
> # I want to be able to be able to `make all` without explicitly 
> listing all # intermediate steps. The first target that fails is # 
> section/section1/group2/step1/do_step
>
> all: \
>       section/section2/section_end \
>
>       echo "$@";
>
> section/section1/section_start: \
>
>       echo "$@";
>
> section/section2/section_start: \
>       section/section1/section_end
>
>       echo "$@";
>
> # eg: section/section1/group1/group_start
> section/%/group1/group_start: \
>
>       echo "$@";
>
> # eg: section/section1/group2/group_start
> section/%/group2/group_start: \
>       section/%/group1/group_end \
>
>       echo "$@";
>
> # eg: section/section1/group1/step1/do_step
> # eg: section/section1/group2/step1/do_step
> section/%/step1/do_step: \
>       section/%/group_start \
>
>       echo "$@";
>
> # eg: section/section1/group1/step2/do_step
> section/%/step2/do_step: \
>       section/%/step1/do_step \
>
>       echo "$@";
>
> # eg: section/section1/group1/step3/do_step
> section/%/step3/do_step: \
>       section/%/step2/do_step \
>
>       echo "$@";
>
> # eg: section/section1/group1/group_end
> section/%/group_end: \
>       section/%/step3/do_step \
>
>       echo "$@";
>
> # eg: section/section1/section_end
> section/%/section_end: \
>       section/%/group2/group_end \
>
>       echo "$@";
>
>
>
> _______________________________________________
> Help-make mailing list
> address@hidden
> https://lists.gnu.org/mailman/listinfo/help-make
>



--
Nicholas Clark
Email: address@hidden
_______________________________________________
Help-make mailing list
address@hidden
https://lists.gnu.org/mailman/listinfo/help-make

_______________________________________________
Help-make mailing list
address@hidden
https://lists.gnu.org/mailman/listinfo/help-make