Re: [Lightning] Lightning frees NULL pointers, and frees others it didn´t allocate

[Paulo César Pereira de Andrade]

2014-07-29 20:48 GMT-03:00 Ian Grant <address@hidden>:
> Dear Paulo

  Hi again Ian,

> You address specific examples I gave, but don't indicate that you understand
> the sense of those comments. Let me try again, because it is important.
>
> If it is to be useful, Lightning should not let particular problems, which
> are accidents of the internal representation it uses, dictate the semantics
> of the interface it exposes to the client applications. _For example:_ If it

  One issue is that lightning offers an interface different than all other
jits around I am aware of. In lighting the programmer actually see hard
registers, besides with a symbolic name, and must understand all
implications of spilling, reloading and clobbering. If one has some
assembly experience it should not be to hard to get used, otherwise,
it may be very difficult to use lightning.
  Other jit libraries hide registers and use "value" opaque objects, and
then make operations on those "value" objects. Those objects may
hold a label, a pointer, a register, a constant, or some kind of internal
state.

  As I kind of commented in the last email, I have been considering
for some time to write a kind of utility library, somewhat like GLU for
OpenGL, for example, glName functions are very low level, need a
complex set of instructions to draw a simple cube, while gluName
functions offer several utility functions to create very complex shapes
with a single call and a few parameters.

> presents an abstraction of JIT state, then that abstraction should not
> 'leak' details of the underlying implementation, because that makes the
> applications that use it dependent on accidental features that could, indeed
> _should_ be changed. The interface, if it is to be effective, must have a
> clear semantics. And if we let internal representation dictate the form of
> that interface, then we will not be able to change the internal
> representation! So it is important that we make these changes as soon as
> possible.

  I tailored lightning 2.0 to attempt to make it as friendly as possible
to create jit for dynamically typed languages. But to truly take advantage
it requires writing function bodies entirely in jit. One of the reasons is that
it forces calling jit as a function, so that it knows "live" bounds and can do
register allocation.

> So I don't need to you to explain to me the reasons why it is like it is, or
> how it actually is. These things don't matter so much. What does matter is
> how we develop it in future. Now none of the changes I suggested necessarily
> effect the application interface _as described._ So I think we could have,
> _for example,_ fully nestable prepare/finish, predictable code size (because
> it is deterministic algorithm with time complexity either linear or  at most
> O(n^2) and fully customisable memory mangement. And these things would make
> the interface easier to document, easier to use, and would not break
> existing applications.

  There aren't many lightning applications so far. But I will do my best to
help in any usage plans :-) Nested prepare/finish is not too difficult, but
would add significant complexity and require breaking the "contract" that
the programmer must know what are or are not callee save registers, and
allow creating very complex conditions inside a prepare/finish sequence
that would be better suited to be handled in an "utility" library.

> My comments regarding the language were directed towards the language in
> which the lightning library itself is written. I am not sure thaI  being
> independent of anything (except the glibc, which is enormous, by the way) is
> necessarily a better thing than providing a well-defined application
> programming interface. And it seems that it is only the lack of good
> recursive data structures that is holding this up.

  Lack of data types is not really the problem. What it depends other
than standard libc is mostly libgcc or same name&functionality symbols,
and in some ports it also requires libm symbols; for example, the arm
port has "fake" floating point registers stored in stack "slots".

> I do not think we need any dumbing down of the API. Certainly not to the
> extent of hiding all register allocation. I fail to see the point of that.
> This thing is not for toy languages. Rather than dumbing it down with a
> higher level interface, we need to smarten-up the lower-level interface so
> that it does what it is supposed ( and what people like me _expect_ it) to
> do. Lightning needs to be good enough that we can generate the entire
> operating system from formal specifications in a functional language. That
> is one of my aims.

  One of the most important missing features is a backend specific pass,
that would be able to combine instructions for the backends that support
it.
  Issues that may cause problems in some use cases are:
1. not providing support to write varargs functions, only to call them
2. lack of an "allocar" call, that is, to dynamically allocate stack space
   with a variable size
3. lack of C++ exceptions support
4. lack of a defined gdb jit interface; cannot do much without a
   symbol/type abstraction
5. lack of a proper way to dump jit to disk and reload, programmer
   must create the abstraction around it to make the jit position
   independent

> This is a very general comment, I am not asking for more information about
> the examples I give, they are examples that I cite to get my more general
> point across.

  Ok. I think it would be very valuable if you could provide some examples
of what you would like to see, for example, what would be a nested
prepare/finish call, or some prototypes or data type definitions.

> Best wishes,
>
> Ian

Thanks,
Paulo