Re: [Qemu-devel] [RFC] test/docker/Makefile.include: add a generic docker-run target

[Alex Bennée]

Fam Zheng <address@hidden> writes:

> On Tue, 09/20 14:56, Alex Bennée wrote:
>> This re-factors the docker makefile to include a docker-run target which
>> can be controlled entirely from environment variables specified on the
>> make command line. This allows us to run against any given docker image
>> we may have in our repository, for example:
>>
>>     make docker-run TEST="test-quick" IMAGE="debian:arm64" \
>>          EXECUTABLE=./aarch64-linux-user/qemu-aarch64
>>
>> The existing address@hidden targets still work but the inline
>> verification has been shunted into other target prerequisites before a
>> sub-make is invoked for the docker-run target.
>
> Hi Alex,
>
> I understand sometimes one can have specialized images, but still: is it
> possible to convert them to Dockerfile and include in the tree?
>
> Or, is this for testing/debugging purpose?

A bit of both. In this particular use case I'm using a debootstrap image
while updating the binfmt_misc executable. Currently there is a 1->N
relationship for debootstrap as we can bootstrap multiple architectures
in different images. By splitting the docker-run from the expansions we
give ourselves a little more flexibility for running stuff.

But I think it's also useful for testing/debugging. I wrote this up as I
was trying to debug a Travis build failure with gcc-6 so I was
generating lots of test images and wanting to build against those. I
would also like to add a travis Dockerfile at some point but at the
moment what exactly goes into one of those is a little opaque to me.

>
>>
>> Signed-off-by: Alex Bennée <address@hidden>
>>
>> ---
>> NB: I dropped the awk magic that verifies the image exists before
>> running. I couldn't get the thing to work in my shell so wasn't quite
>> sure what it was doing.
>
> It was to allow "make docker-test" to skip debian-bootstrap image if it is not
> there (e.g. when qemu-user not available).

Ahh ok. I got a little confused as the docker images command can filter
things based on tag so maybe we can come up with a cleaner test?

>
> I'm not much too concerned about that though, since most of the time we will
> use address@hidden, for specific combinations, instead of docker-test for a
> blanket coverage.

What does patchew use?

>
>> ---
>>  tests/docker/Makefile.include | 82 
>> ++++++++++++++++++++++++++++++++-----------
>>  1 file changed, 62 insertions(+), 20 deletions(-)
>>
>> diff --git a/tests/docker/Makefile.include b/tests/docker/Makefile.include
>> index 19d4cc7..ee47085 100644
>> --- a/tests/docker/Makefile.include
>> +++ b/tests/docker/Makefile.include
>> @@ -101,31 +101,73 @@ docker:
>>      @echo '    NOCACHE=1            Ignore cache when build images.'
>>      @echo '    EXECUTABLE=<path>    Include executable in image.'
>>
>> -docker-run-%: CMD = $(shell echo '$@' | sed -e 
>> 's/docker-run-\(address@hidden)@\(.*\)/\1/')
>> -docker-run-%: IMAGE = $(shell echo '$@' | sed -e 
>> 's/docker-run-\(address@hidden)@\(.*\)/\2/')
>> -docker-run-%: docker-qemu-src
>> +# This rule if for directly running against an arbitary docker target.
>> +# It is called by the expanded docker targets (e.g. make
>> +# address@hidden) which will do additional verification.
>> +#
>> +# For example: make docker-run TEST="test-quick" IMAGE="debian:arm64" 
>> EXECUTABLE=./aarch64-linux-user/qemu-aarch64
>> +#
>> +docker-run: docker-qemu-src
>
> This target should probably be documented in "make docker".

OK

>
>>      @mkdir -p "$(DOCKER_CCACHE_DIR)"
>> -    @if test -z "$(IMAGE)" || test -z "$(CMD)"; \
>> -            then echo "Invalid target"; exit 1; \
>> +    @if test -z "$(IMAGE)" || test -z "$(TEST)"; \
>> +            then echo "Invalid target $(IMAGE)/$(TEST)"; exit 1; \
>>      fi
>> -    $(if $(filter $(TESTS),$(CMD)),$(if $(filter $(IMAGES),$(IMAGE)), \
>> -            $(call quiet-command,\
>> -                    if $(SRC_PATH)/tests/docker/docker.py images | \
>> -                            awk '$$1=="qemu" && $$2=="$(IMAGE)"{found=1} 
>> END{exit(!found)}'; then \
>> -                            $(SRC_PATH)/tests/docker/docker.py run $(if 
>> $V,,--rm) \
>> -                            -t \
>> -                            $(if $(DEBUG),-i,--net=none) \
>> -                            -e TARGET_LIST=$(TARGET_LIST) \
>> +    $(if $(EXECUTABLE),                                             \
>> +            $(call quiet-command,                                   \
>> +                    $(SRC_PATH)/tests/docker/docker.py update       \
>> +                    $(IMAGE) $(EXECUTABLE)))
>> +    $(call quiet-command,                                           \
>> +            $(SRC_PATH)/tests/docker/docker.py run                  \
>> +                            -t                                      \
>> +                            $(if $V,,--rm)                          \
>> +                            $(if $(DEBUG),-i,--net=none)            \
>> +                            -e TARGET_LIST=$(TARGET_LIST)           \
>>                              -e EXTRA_CONFIGURE_OPTS=$(EXTRA_CONFIGURE_OPTS) 
>> \
>> -                            -e V=$V -e J=$J -e DEBUG=$(DEBUG)\
>> -                            -e CCACHE_DIR=/var/tmp/ccache \
>> +                            -e V=$V -e J=$J -e DEBUG=$(DEBUG)       \
>> +                            -e CCACHE_DIR=/var/tmp/ccache           \
>>                              -v $$(readlink -e 
>> $(DOCKER_SRC_COPY)):/var/tmp/qemu:z$(COMMA)ro \
>>                              -v $(DOCKER_CCACHE_DIR):/var/tmp/ccache:z \
>> -                            qemu:$(IMAGE) \
>> -                            /var/tmp/qemu/run \
>> -                            $(CMD); \
>> -                    fi \
>> -                    , "  RUN $(CMD) in $(IMAGE)")))
>> +                            $(IMAGE)                                \
>> +                            /var/tmp/qemu/run                       \
>> +                            $(TEST), "  RUN $(TEST) in ${IMAGE}")
>> +
>> +#
>> +# Verification targets
>> +#
>> +# These targets help verify the test (CMD) and docker tag (IMAGE) are
>> +# part of the built in set of tests and images. You can still call the
>> +# docker-run target directly for testsing against arbitary images.
>
> s/arbitary/arbitrary/
>
> One more of this below.

OK.

>
>> +#
>> +
>> +docker-verify-image-%: IMAGE = $(shell echo '$@' | sed -e 
>> 's/docker-verify-image-\(address@hidden)@\(.*\)/\2/')
>> +docker-verify-image-%:
>> +    @if test -z "$(IMAGE)";                         \
>> +            then echo "Invalid image"; exit 1;      \
>> +    fi
>> +    $(if $(findstring $(IMAGE), $(DOCKER_IMAGES))   \
>
> This seems inaccurate. what if it is in a form of "$(findstring abc,
> abcde)"?

I'll see if I can replace it with the original filter test.

>
>> +            , @echo "Verified $(IMAGE)"             \
>> +            , @echo "$(IMAGE) is not a known image"; exit 1)
>> +
>> +docker-verify-test-%: CMD = $(shell echo '$@' | sed -e 
>> 's/docker-verify-test-\(address@hidden)@\(.*\)/\1/')
>> +docker-verify-test-%:
>> +    @if test -z "$(CMD)";                           \
>> +            then echo "Invalid test"; exit 1;       \
>> +    fi
>> +    $(if $(findstring $(CMD), $(DOCKER_TESTS))      \
>
> The same question to findstring as above.
>
>> +            , @echo "Verified $(CMD)"               \
>
> No need to echo anything in case of verified, or at least use quiet-command
> please.

Sure. This was mostly convincing me I got my make-fu right ;-)

>
>> +            , @echo "$(CMD) is not a known test"; exit 1)
>> +
>> +# Run targets
>> +#
>> +# This will take a target such as address@hidden and verify that:
>> +#  - the test test-foo is a known test
>> +#  - the image bar is a known image
>> +#
>> +# It will then call the docker-run
>> +docker-run-%: CMD = $(shell echo '$@' | sed -e 
>> 's/docker-run-\(address@hidden)@\(.*\)/\1/')
>> +docker-run-%: IMAGE = $(shell echo '$@' | sed -e 
>> 's/docker-run-\(address@hidden)@\(.*\)/\2/')
>> +docker-run-%: docker-verify-image-% docker-verify-test-%
>> +    @make docker-run TEST=$(CMD) IMAGE=qemu:$(IMAGE)
>>
>>  docker-clean:
>>      $(call quiet-command, $(SRC_PATH)/tests/docker/docker.py clean)
>> --
>> 2.9.3
>>
>
> Fam


--
Alex Bennée