Re: Separate build rules (was Re: [Mingw-cross-env-list] Mingw64)

[Tony Theodore]

Hi Volker,

Thanks for the long reply, I finally found some time to think about this again.

On 17 September 2010 21:10, Volker Grabsch <address@hidden> wrote:
[...]
> I thought that for the first step of simply using mingw-w64 to
> build 32-bit-only, this wouldn't be necessary. It is important
> to reach complete 32-bit support with mingw-w64 first, without
> being distracted by 64-bit details that aren't relevant yet.

I think complete 32-bit support is still a fair way off for mingw-w64.
I'd rather see continued 32-bit support using mingw.org and a subset
of 64-bit packages.

> Having said that, here's my current plan.
>
> (I'll introduce the new stuff in Mingw-cross-env and convert all
>  *.mk files if you really need that for your actual work now.)

Thanks, I'm in no need of this in the near future and I think it will
be years before it becomes a major issue.

> First, we replace $(TARGET) with $(TARGETS):
>
>    TARGETS := i686-... x86_64-...
>
> Then, each src/*.mk file contains multiple $(PKG)_BUILD rules:
>
>    define $(PKG)_BUILD_i686
>        ...
>    endef
>
>    define $(PKG)_BUILD_x86_64
>        ...
>    endef
>
> By default, all 32-bit versions and all 64-bit versions of
> all packages are built. They already have separate directories
> (.../TARGET/lib/ etc), so the only danger is name clashing in
> the .../bin/ directory, but that is already solved by our
> requirement that everything in .../bin/ must have a prefix.

GCC also installs to ../lib and ../libexec, but they are also properly prefixed.

> 32-bit-only or 64-bit-only builds are possibile by adjusting
> TARGETS accordingly, but that's optimization. By default we
> simply build and test both, and everything runs fine.

Initially, I'd probably leave 32-bit only as the default, but as you
say, this is an optimistaion. This seems to be a better long-term
solution and would allow future targets (say OSX or shared builds) to
be cleanly separated.

> The "install"-markers get a CPU suffix:
>
>    .../installed/mypackage
> ->
>    .../installed/mypackage_i686
>    .../installed/mypackage_x86_64
>
> There might be other places to be adjusted, too.

Maybe ./log becomes ./log-$(TARGET) ?

> Packages without 64-bit support have an empty BUILD_x86_64 rule.

I imagine the empty rules will always succeed so the install marker
will be written.

> Packages that have identical rules for 32/64 bit can be written as:
>
>    define $(PKG)_BUILD_i686
>        ...
>    endef
>
>    define $(PKG)_BUILD_x86_64
>        $(mypackage_BUILD_i686)
>    endef
>
> Note that you _can't_ use "$($(PKG)_BUILD_i686)" here yet. [1]
>
> We could also introduce a special rule in the Makefile, such
> that it tries $(PKG)_BUILD before trying specialized stuff.
> That way, packages could simply write in the old syntax for
> packages with identical build rules:
>
>    define $(PKG)_BUILD
>        ...
>    endef
>
> But I think the explicit approach mentioned above would be
> cleaner.

Agreed, we can always use some variables for common parts:

    $(PKG)_CONFIGURE_OPTIONS := ...

> Another possibility would be the introduction of conditionals
> into the build rules, like this:
>
>    define $(PKG)_BUILD
>        ... code for both ...
>        $(if $(filter i686-%, $(TARGET)),
>                              # We might have to use $(3)
>                              # instead of $(TARGET)
>            ... code for 32-bit ...
>        ,
>            ... code for 64-bit ...
>        )
>        ... code for both ...
>    endef
>
> However, I doubt that this would increase readability. I think
> it is really a good idea to keep and develop the build rules
> separate from each other. It might be useful for cases where
> the difference between 32-bit and 64-bit is just a minor tweak,
> though.

Keeping them separate is a good idea, we can always (I think) do things like:

    define $(PKG)_BUILD_x86_64
        ...some $(SED) replacement..
        $(mypackage_BUILD_i686)
    endef

    define $(PKG)_BUILD_x86_64
        $(subst mingw,mingw64,$(mypackage_BUILD_i686))
    endef

> I more critical point are the dependencies. I thougt about
> having separate dependencies for each package:
>
>    $(PKG)_DEPS_i686   := ...
>    $(PKG)_DEPS_x86_64 := ...
>
> However, I guess this _wouldn't_ be a good idea. Since we
> use empty build rules for unsupported packages, mentioning
> them in the DEPS doesn't do any harm. That's why we can
> simply keep a complete list of dependencies for each package.

The two scenarios I imagine can be easily handled by empty build rules.

1. GCC in my scenario of multiple runtimes. Simply have empty build
x86_64 rules for mingwrt and w32api, and an empty i686 rule for
mingw-w64.

2. An optional dependency fails to build (eg postgresql for qt). The
postgresql x86_64 build rule will be empty and the psql configuration
removed from the qt build rule. Packages that auto-detect
(openscenegraph/ffmpeg) should have no problems - simply an empty
ffmpeg rule.

> I also thought about the possibility of having separate files
> for each package, which would allow for separate versions and
> dependencies for the 32/64-bit version of each package. However,
> this would effectively double that effort needed to keep
> Mingw-cross-env up to date. A similar issue arose with the
> static/shared variants which is why we stick with one of them.
>
> So our goal should be to keep the 32/64-bit versions as
> closely together as possible, using the same versions
> with the same dependencies and the same set of patches.
> (Patches should contain only compatibility stuff anyway.)

I agree, multiple files would be very difficult to manage, and the
closer we can keep things (while leaving the flexibility to separate
when need) is the best option.

Cheers,

Tony