Re: [PATCH 6/7] gitlab-ci: Determine the number of jobs dynamically

[Thomas Huth]

On 28/05/2020 10.41, Alex Bennée wrote:
> 
> Thomas Huth <thuth@redhat.com> writes:
> 
>> Some people might want to run the gitlab CI pipelines in an environment
>> where multiple CPUs are available to the runners, so let's rather get
>> the number for "-j" from the "nproc" program (increased by 1 to compensate
>> for jobs that wait for I/O) instead of hard-coding it.
>>
>> Signed-off-by: Thomas Huth <thuth@redhat.com>
> <snip>
>> @@ -25,8 +27,8 @@ build-system1:
>>   - ../configure --enable-werror --target-list="aarch64-softmmu alpha-softmmu
>>        cris-softmmu hppa-softmmu lm32-softmmu moxie-softmmu 
>> microblazeel-softmmu
>>        mips64el-softmmu m68k-softmmu ppc-softmmu riscv64-softmmu 
>> sparc-softmmu"
>> - - make -j2
>> - - make -j2 check
>> + - make -j"$JOBS"
>> + - make -j"$JOBS" check
>>  
>>  build-system2:
>>   image: fedora:latest
>> @@ -40,8 +42,8 @@ build-system2:
>>   - ../configure --enable-werror --target-list="tricore-softmmu 
>> unicore32-softmmu
>>        microblaze-softmmu mips-softmmu riscv32-softmmu s390x-softmmu 
>> sh4-softmmu
>>        sparc64-softmmu x86_64-softmmu xtensa-softmmu nios2-softmmu 
>> or1k-softmmu"
>> - - make -j2
>> - - make -j2 check
>> + - make -j"$JOBS"
>> + - make -j"$JOBS" check
>>  
>>  build-disabled:
>>   image: fedora:latest
>> @@ -56,8 +58,8 @@ build-disabled:
>>        --disable-qom-cast-debug --disable-spice --disable-vhost-vsock
>>        --disable-vhost-net --disable-vhost-crypto --disable-vhost-user
>>        --target-list="i386-softmmu ppc64-softmmu mips64-softmmu 
>> i386-linux-user"
>> - - make -j2
>> - - make -j2 check-qtest SPEED=slow
>> + - make -j"$JOBS"
>> + - make -j"$JOBS" check-qtest SPEED=slow
> 
> I would make all the check jobs use a single core as it otherwise gets
> hard to figure out exactly where something broke/hung.

It's a somewhat double-edged sword ... either faster CI test times, or
more deterministic output ... so far I didn't suffer the problem with
the deterministic output in the gitlab-CI yet (unlike with Travis), so
I'd rather keep the -j here for now. We can still remove it later if we
hit a bug that is hard to debug otherwise.

 Thomas