Re: [PATCH] build: pull: Compile .scm files in one process.

[Ludovic Courtès]

address@hidden (Taylan Ulrich "Bayırlı/Kammer") skribis:

> address@hidden (Ludovic Courtès) writes:

[...]

>> I think we’re in trouble if DEBUG-PORT is a real port because ports
>> aren’t thread-safe in 2.0, and ‘compile-file’ might write to it.  Void
>> ports seem to be OK because their ‘write’ method doesn’t do anything.
>>
>> So I think we have to do things sequentially when DEBUG-PORT is
>> non-void.
>>
>> I’m also concerned about modules accessed by concurrent calls to
>> ‘compile-file’ because modules and hash tables aren’t thread-safe
>> either.
>>
>> WDYT?
>>
>> Thanks to the two of you!  Sounds like we’ll soon have a less slow or
>> even a fast ‘guix pull’.  ;-)
>
> Hmm, how about this one?  I guess it could still garble output if
> 'compile-file' writes things in small pieces that belong together, but
> at least it's sure not to crash.  Compile-file doesn't seem to write a
> lot either; one probably won't see much garbled output in practice.
> Maybe the speed gain makes this acceptable.

It’s not about garbled output; it’s about concurrent modifications of
the port’s internal structure, which can lead to segfaults and whatnot.

> Re. modules, AFAIUI compile-file shouldn't do any further mutations on
> modules, since we load them in advance.  Maybe Andy can tell how safe
> this is.

Right, it *seems* reasonable.  Andy, Mark?

> From 78be6d09d2d4c0a563be14c66ac2a1a345ff9b1d Mon Sep 17 00:00:00 2001
> From: =?UTF-8?q?Taylan=20Ulrich=20Bay=C4=B1rl=C4=B1/Kammer?=
>  <address@hidden>
> Date: Thu, 5 Nov 2015 23:43:20 +0100
> Subject: [PATCH] build: pull: Compile .scm files in one process.
>
> * guix/build/pull.scm (call-with-process, report-build-progress)
> (p-for-each): Remove.
> (thread-safe-port): New procedure.
> (build-guix): Load and compile files in one process.

[...]

> +(define (thread-safe-port port)
> +  (define mutex (make-mutex))
> +  (define (wrap proc)
> +    (lambda args
> +      (with-mutex mutex (apply proc args))))
> +  (make-soft-port
> +   (vector (wrap write-char) (wrap display) (wrap force-output) #f #f) "w"))

This doesn’t address the fact that internally, things may still go wrong
(see ‘scm_lfwrite’ and the likes.)

I think we simply need to build sequentially when ports are involved,
which is when --debug is used.

Could you address this last bit?

Thanks!

Ludo’.