[Date Prev][Date Next][Thread Prev][Thread Next][Date Index][Thread Index]
Re: [Qemu-devel] [PATCH v5 45/49] tests/tcg/Makefile: update to be calle
From: |
Alex Bennée |
Subject: |
Re: [Qemu-devel] [PATCH v5 45/49] tests/tcg/Makefile: update to be called from Makefile.target |
Date: |
Tue, 29 May 2018 13:49:29 +0100 |
User-agent: |
mu4e 1.1.0; emacs 26.1.50 |
Philippe Mathieu-Daudé <address@hidden> writes:
> Hi Alex,
>
> On 05/25/2018 11:19 AM, Alex Bennée wrote:
>> This make is now invoked from each individual target make with the
>> appropriate CC and EXTRA_CFLAGS set for each guest. It then includes
>> additional Makefile.targets from:
>>
>> - tests/tcg/multiarch (always)
>> - tests/tcg/$(TARGET_BASE_ARCH) (if available)
>> - tests/tcg/$(TARGET_NAME)
>>
>> The order is important as the later Makefile's may want to suppress
>> TESTS from its base arch profile. Each included Makefile.target is
>> responsible for adding TESTS as well as defining any special build
>> instructions for individual tests.
>>
>> Signed-off-by: Alex Bennée <address@hidden>
>> Reviewed-by: Richard Henderson <address@hidden>
>> Reviewed-by: Philippe Mathieu-Daudé <address@hidden>
>> Tested-by: Philippe Mathieu-Daudé <address@hidden>
>> ---
>> v2
>> - cleaner approach to include sub makefiles
>> - move TESTS/VPATH manipulation into sub-makefile
>> - avoid double inclusion when TARGET_BASE_ARCH==TARGET_NAME
>> v4
>> - add timeout to default runner
>> - clean-up comments about build flags
>> - update to handle BUILD_STATIC
>> - add MAINTAINERS entry
>> v5
>> - support EXTRA_RUNS for run variants
>> ---
>> MAINTAINERS | 5 ++
>> tests/tcg/Makefile | 179 ++++++++++++++++++---------------------------
>> 2 files changed, 76 insertions(+), 108 deletions(-)
>>
>> diff --git a/MAINTAINERS b/MAINTAINERS
>> index ec385a15df..bfa8d0fb9c 100644
>> --- a/MAINTAINERS
>> +++ b/MAINTAINERS
>> @@ -2138,6 +2138,11 @@ W: https://travis-ci.org/qemu/qemu
>> W: https://app.shippable.com/github/qemu/qemu
>> W: http://patchew.org/QEMU/
>>
>> +Guest Test Compilation Support
>> +M: Alex Bennée <address@hidden>
>> +F: tests/tcg/Makefile
>> +L: address@hidden
>> +
>> Documentation
>> -------------
>> Build system architecture
>> diff --git a/tests/tcg/Makefile b/tests/tcg/Makefile
>> index e12395117a..8158c1f099 100644
>> --- a/tests/tcg/Makefile
>> +++ b/tests/tcg/Makefile
>> @@ -1,125 +1,88 @@
>> +# -*- Mode: makefile -*-
>> +#
>> +# TCG tests
>> +#
>> +# These are complicated by the fact we want to build them for guest
>> +# systems. This requires knowing what guests we are building and which
>> +# ones we have cross-compilers for or docker images with
>> +# cross-compilers.
>> +#
>> +# The tests themselves should be as minimal as possible as
>> +# cross-compilers don't always have a large amount of libraries
>> +# available.
>> +#
>> +# We only include the host build system for SRC_PATH and we don't
>> +# bother with the common rules.mk. We expect the following:
>> +#
>> +# CC - the C compiler command
>> +# EXTRA_CFLAGS - any extra CFLAGS
>> +# BUILD_STATIC - are we building static binaries
>> +#
>> +# By default all tests are statically compiled but some host systems
>> +# may not package static libraries by default. If an external
>> +# cross-compiler can only build dynamic libraries the user might need
>> +# to make extra efforts to ensure ld.so can link at runtime when the
>> +# tests are run.
>> +#
>> +# We also accept SPEED=slow to enable slower running tests
>> +#
>> +# We also expect to be in the tests build dir for the FOO-linux-user.
>> +#
>> +
>> -include ../../config-host.mak
>> --include $(SRC_PATH)/rules.mak
>> +-include ../config-target.mak
>>
>> -$(call set-vpath, $(SRC_PATH)/tests/tcg)
>> +quiet-command = $(if $(V),$1,$(if $(2),@printf " %-7s %s\n" $2 $3 && $1,
>> @$1))
>>
>> -QEMU=../../i386-linux-user/qemu-i386
>> -QEMU_X86_64=../../x86_64-linux-user/qemu-x86_64
>> -CC_X86_64=$(CC_I386) -m64
>> +# Tests we are building
>> +TESTS=
>>
>> -QEMU_INCLUDES += -I../..
>> -CFLAGS=-Wall -O2 -g -fno-strict-aliasing
>> -#CFLAGS+=-msse2
>> +# Start with a blank slate, the build targets get to add stuff first
>> +CFLAGS=
>> +QEMU_CFLAGS=
>> LDFLAGS=
>>
>> -# TODO: automatically detect ARM and MIPS compilers, and run those too
>> -
>> -# runcom maps page 0, so it requires root privileges
>> -# also, pi_10.com runs indefinitely
>> -
>> -I386_TESTS=hello-i386 \
>> - sha1-i386 \
>> - test-i386 \
>> - test-i386-fprem \
>> - # runcom
>> +# The QEMU for this TARGET
>> +QEMU=../qemu-$(TARGET_NAME)
>> +TIMEOUT=15
>>
>> -# native i386 compilers sometimes are not biarch. assume cross-compilers
>> are
>> -ifneq ($(ARCH),i386)
>> -I386_TESTS+=run-test-x86_64
>> +# The order we include is important. We include multiarch, base arch
>> +# and finally arch if it's not the same as base arch.
>> +-include $(SRC_PATH)/tests/tcg/multiarch/Makefile.target
>> +-include $(SRC_PATH)/tests/tcg/$(TARGET_BASE_ARCH)/Makefile.target
>> +ifneq ($(TARGET_BASE_ARCH),$(TARGET_NAME))
>> +-include $(SRC_PATH)/tests/tcg/$(TARGET_NAME)/Makefile.target
>> endif
>>
>> -TESTS = test_path
>> -ifneq ($(call find-in-path, $(CC_I386)),)
>> -TESTS += $(I386_TESTS)
>> +# Add the common build options
>> +CFLAGS+=-Wall -O0 -g -fno-strict-aliasing
>> +ifeq ($(BUILD_STATIC),y)
>> +LDFLAGS+=-static
>> endif
>>
>> -all: $(patsubst %,run-%,$(TESTS))
>> -test: all
>> -
>> -# rules to run tests
>> -
>> -.PHONY: $(patsubst %,run-%,$(TESTS))
>> -
>> -run-%: %
>> - -$(QEMU) ./$*
>> -
>> -run-hello-i386: hello-i386
>> -run-sha1-i386: sha1-i386
>> -
>> -run-test-i386: test-i386
>> - ./test-i386 > test-i386.ref
>> - -$(QEMU) test-i386 > test-i386.out
>> - @if diff -u test-i386.ref test-i386.out ; then echo "Auto Test OK"; fi
>> -
>> -run-test-i386-fprem: test-i386-fprem
>> - ./test-i386-fprem > test-i386-fprem.ref
>> - -$(QEMU) test-i386-fprem > test-i386-fprem.out
>> - @if diff -u test-i386-fprem.ref test-i386-fprem.out ; then echo "Auto
>> Test OK"; fi
>> -
>> -run-test-x86_64: test-x86_64
>> - ./test-x86_64 > test-x86_64.ref
>> - -$(QEMU_X86_64) test-x86_64 > test-x86_64.out
>> - @if diff -u test-x86_64.ref test-x86_64.out ; then echo "Auto Test OK";
>> fi
>> -
>> +%: %.c
>> + $(CC) $(CFLAGS) $(EXTRA_CFLAGS) $< -o $@ $(LDFLAGS)
>>
>> -run-runcom: runcom
>> - -$(QEMU) ./runcom $(SRC_PATH)/tests/pi_10.com
>> +all: $(TESTS)
>>
>> -run-test_path: test_path
>> - ./test_path
>> +#
>> +# Test Runners
>> +#
>> +# By default we just run the test with the appropriate QEMU for the
>> +# target. More advanced tests may want to override the runner in their
>> +# specific make rules. Additional runners for the same binary should
>> +# be added to EXTRA_RUNS.
>> +#
>>
>> -# rules to compile tests
>> +RUN_TESTS=$(patsubst %,run-%, $(TESTS))
>> +RUN_TESTS+=$(EXTRA_RUNS)
>>
>> -hello-i386: hello-i386.c
>> - $(CC_I386) -nostdlib $(CFLAGS) -static $(LDFLAGS) -o $@ $<
>> - strip $@
>> -
>> -# i386/x86_64 emulation test (test various opcodes) */
>> -test-i386: test-i386.c test-i386-code16.S test-i386-vm86.S \
>> - test-i386.h test-i386-shift.h test-i386-muldiv.h
>> - $(CC_I386) $(QEMU_INCLUDES) $(CFLAGS) $(LDFLAGS) -o $@ \
>> - $(<D)/test-i386.c $(<D)/test-i386-code16.S
>> $(<D)/test-i386-vm86.S -lm
>> -
>> -test-i386-fprem: test-i386-fprem.c
>> - $(CC_I386) $(QEMU_INCLUDES) $(CFLAGS) $(LDFLAGS) -o $@ $^
>> -
>> -test-x86_64: test-i386.c \
>> - test-i386.h test-i386-shift.h test-i386-muldiv.h
>> - $(CC_X86_64) $(QEMU_INCLUDES) $(CFLAGS) $(LDFLAGS) -o $@
>> $(<D)/test-i386.c -lm
>> -
>> -# vm86 test
>> -runcom: runcom.c
>> - $(CC_I386) $(CFLAGS) $(LDFLAGS) -o $@ $<
>> -
>> -# speed test
>> -sha1-i386: sha1.c
>> - $(CC_I386) $(CFLAGS) $(LDFLAGS) -o $@ $<
>> -
>> -# arm test
>> -hello-arm: hello-arm.o
>> - arm-linux-ld -o $@ $<
>> -
>> -hello-arm.o: hello-arm.c
>> - arm-linux-gcc -Wall -g -O2 -c -o $@ $<
>> -
>> -test-arm-iwmmxt: test-arm-iwmmxt.s
>> - cpp < $< | arm-linux-gnu-gcc -Wall -static -march=iwmmxt -mabi=aapcs -x
>> assembler - -o $@
>> -
>> -# MIPS test
>> -hello-mips: hello-mips.c
>> - mips-linux-gnu-gcc -nostdlib -static -mno-abicalls -fno-PIC -mabi=32
>> -Wall -Wextra -g -O2 -o $@ $<
>> -
>> -hello-mipsel: hello-mips.c
>> - mipsel-linux-gnu-gcc -nostdlib -static -mno-abicalls -fno-PIC -mabi=32
>> -Wall -Wextra -g -O2 -o $@ $<
>> -
>> -# testsuite for the CRIS port.
>> -test-cris:
>> - $(MAKE) -C cris check
>> +run-%: %
>> + $(call quiet-command, \
>> + timeout $(TIMEOUT) $(QEMU) $< > $<.out, \
>> + "TEST", "$< on $(TARGET_NAME)")
>
> Should we use --preserve-status here?
>
> I'm getting Error 124:
>
> If the command times out, and --preserve-status is not set,
> then exit with status 124. Otherwise, exit with the status of
> COMMAND. If no signal is specified, send the TERM signal upon
> timeout. The TERM signal kills any process that does not block
> or catch that signal. It may be necessary to use the KILL (9)
> signal, since this signal cannot be caught, in which case the
> exit status is 128+9 rather than 124.
I guess it only really matters if the test cases are going to catch and
handle TERM signals. Currently I think any non-zero response is enough
to exit make.
>
> $ make -j1 run-tcg-tests-i386-linux-user
> [...]
> RUN-TESTS for i386
> TEST test-mmap (default) on i386
> TEST sha1 on i386
> TEST linux-test on i386
> TEST testthread on i386
> TEST test-i386 on i386
> TEST hello-i386 on i386
> SKIPPED test-i386-fprem because SLOW
> TEST runcom on i386
> make[2]: *** [/src/qemu/tests/tcg/i386/Makefile.target:34: run-runcom]
> Error 124
> make[1]: *** [/src/qemu/tests/tcg/Makefile.include:71: run-guest-tests]
> Error 2
> make: *** [/src/qemu/tests/Makefile.include:931:
> run-tcg-tests-i386-linux-user] Error 2
>
>>
>> -# testsuite for the LM32 port.
>> -test-lm32:
>> - $(MAKE) -C lm32 check
>> +.PHONY: run
>> +run: $(RUN_TESTS)
>>
>> -clean:
>> - rm -f *~ *.o test-i386.out test-i386.ref \
>> - test-x86_64.log test-x86_64.ref qruncom $(TESTS)
>> +# There is no clean target, the calling make just rm's the tests build dir
>>
--
Alex Bennée
- [Qemu-devel] [PATCH v5 49/49] tests/tcg: add run, diff, and skip helper macros, (continued)
- [Qemu-devel] [PATCH v5 49/49] tests/tcg: add run, diff, and skip helper macros, Alex Bennée, 2018/05/25
- [Qemu-devel] [PATCH v5 46/49] Makefile.target: add (clean-/build-)guest-tests targets, Alex Bennée, 2018/05/25
- [Qemu-devel] [PATCH v5 43/49] docker: move debian-powerpc-cross to sid based build, Alex Bennée, 2018/05/25
- [Qemu-devel] [PATCH v5 44/49] tests/tcg: enable building for PowerPC, Alex Bennée, 2018/05/25
- [Qemu-devel] [PATCH v5 48/49] tests/tcg: override runners for broken tests, Alex Bennée, 2018/05/25
- [Qemu-devel] [PATCH v5 47/49] tests/Makefile.include: add [build|clean|check]-tcg targets, Alex Bennée, 2018/05/25
- [Qemu-devel] [PATCH v5 45/49] tests/tcg/Makefile: update to be called from Makefile.target, Alex Bennée, 2018/05/25
[Qemu-devel] [PATCH v5.1 21/49] tests/tcg/i386: add runner for test-i386-fprem, Alex Bennée, 2018/05/25
Re: [Qemu-devel] [PATCH v5 00/49] fix buildings of tests/tcg, Alex Bennée, 2018/05/25