Re: GNU Lightning needs a new maintainer

[Paul Cercueil]

Hi Paulo, great to hear from you.

Le lun., mai 9 2022 at 13:53:33 -0300, Paulo César Pereira de Andrade 
<paulo.cesar.pereira.de.andrade@gmail.com> a écrit :
> ]Em qua., 13 de abr. de 2022 às 15:43, Paul Cercueil
> <paul@crapouillou.net> escreveu:
> >  Hi,
>
>   Hi,
>
> >  We are a few people trying to contribute to GNU Lightning, but the
> >  maintainer (Paulo) seems to be MIA. His last email on the mailing 
> > list
> >  was on May of last year. Since then we couldn't get a patch in.
>
>   Sorry. I understand the situation. I missed most emails, and only
> look at this email address every few weeks or months. I will attempt
> to reply next to at least a few from archives.
>
>   My job and my real life are absorbing most of my time, and very few
> time is left for volunteer efforts.

That's understandable. The problem is that you are the only one who can 
accept patches into the repository... Maybe we could have a 
co-maintainer?

> >  I think it would make sense to find a new maintainer to unlock the
> >  situation.
> >
> >  The second point I would like to talk about: I do wish this project 
> > was
> >  on github/gitlab, one part for the convenience (managing PRs and
> >  integrated bugzilla), and the other part is the need of CI. GNU
> >  Lightning supports 10+ architectures in various configurations, it 
> > is
> >  just impossible to test properly every patch that we make. Automated
> >  builds would be a big help.
> >
> >  Cheers,
> >  -Paul
>
> Thanks!
> Paulo
>
>
> From more recent to older:
>
> [PATCH] Add conditional move opcodes:
> I absolutely do not dislike it. But this might be an optimization in 
> the
> wrong place. Either lightning should understand and create it, but 
> need
> a back end specific optimizer, or this is just the start of making it 
> far
> more complex, and would need a fallback for arches that do not have
> it, and it might be complex, would be like jit_b{o,x}{add,sub}* but 
> instead
> of a conditional jump, a conditional move.
> There are a few optimizations, and this likely could be done in the
> generic optimizer, if it knows the target extra features, or, it 
> could be
> done by a backend specific optimization pass. But really, it would be
> better to think on how make lightning simpler, not more complicated.

Some time ago I did update the patch, the latest can be found here:
https://github.com/pcercuei/gnu_lightning/commit/6f101bf8eccef737d60bf7e6ba85558db49e7908

I added conditional opcodes for ARM and Aarch64, and a fallback 
implementation for PowerPC. Adding fallback implementations for the 
rest of the architectures shouldn't be too hard, the hard part is 
creating a qemu image and testing for every arch...

If I understand you correctly, the optimizer would detect this pattern:

branch = jit_beqi(r2, 0);
jit_movr(r0, r1);
jit_patch(branch);

And replace it with a conditional opcode for the current arch?

I guess that works for me, it keeps the API simple, but I'm not sure I 
can implement that. The inner Lightning (outside *-cpu.c) is scary and 
I wouldn't know where to start.

As for making Lightning more complex... there are a few opcodes that I 
think are really missing, for instance endian-swap opcodes (htonr/ntohr 
only do something on little-endian), and maybe a "count leading zeros" 
opcode. Also a jit_dead() similar to jit_live(). For the rest I'm 
pretty happy with what Lightning provides.

> [Patches for configure Script and Doc]
> Sorry about these. I do not use Ubuntu but will add the patches and
> possibly patch it to not break at least in the environments I use.
>
> [[PATCH 0/6] Misc patches to pass the testsuite and remove warnings]
> Can work on some of these, but really, -fno-strict-aliasing should be
> default if doing any optimized build, or, you might end with a hidden 
> bug
> very difficult to find. Better to just stick with 
> -fno-strict-aliasing.
>
> [Issue when using jit_label() before a jump or branch]
> It appears it was already discussed in the thread. Unless I 
> misunderstood
> the issue, the generated code is correct. It is a non conditional 
> jump to
> a non conditional jump, so, it is ok to take a short circuit.
>
> [[PATCH 1/6] riscv: Fix movi/relatives loading incorrect value]
> The patch is incorrect. The sign extension is expected. The "magic" is
> the use of addiw. Note that constants are 64 bit, so, you might be
> doing something wrong.
>
> [[PATCH 2/6] arm: Force RES0 bits in TST/A1 to be 0]
> Need to check it on actual hardware. For the past 10+ years I did only
> test arm variants on qemu images.
>
> [[PATCH 3/6] mips: Use DEXT when in 64-bit mode instead of EXT]
> I did never test on mips 64. Note that apparently will just amend a
> patch you already did send. But I can add it.
>
> [[PATCH 6/6] Include math.h in check/lightning.c]
> This is straight and should not have any side effects.
>
> [[PATCH 5/6] misc: Add bfd_arch to arm/aarch64/mips]
> This appears to dependent on the binutils version. As long as it works
> with older binutils (from my test vms) it can be added, otherwise, 
> will
> need a binutils version check.

The binutils version check is broken, it always detects wrong here.

By the way, I have this alternative patch that greatly simplifies the 
BFD init code:
https://github.com/pcercuei/gnu_lightning/commit/08cfdf131fa5cd772cba48e88f68dc00a7fd0a3d

> [[PATCH 4/6] ppc: Add endianness check to _truncr_d_i]
> Thanks, this appears to be correct, and somehow missed when doing
> the ppc64le port. Need to review again, probably I missed a make check
> in some of the last steps.
>
> [Atomic operations]
> If patches are provided, sure it would be welcome. My main usage of
> lightning was fine with calls to pthread_* as it was only jit for a 
> custom
> language. The main problem comes with fallbacks for arches that do
> not support it, so, only some minimal set likely could be added.

Fallbacks mean using real locking, which means calls to C, is that 
something we really want?

Cheers,
-Paul

> From here I see my comments... but will review further the archives.