[Top][All Lists]

[Date Prev][Date Next][Thread Prev][Thread Next][Date Index][Thread Index]

Re: [lwip-devel] Proposals for lwip

From: address@hidden
Subject: Re: [lwip-devel] Proposals for lwip
Date: Tue, 16 Jul 2019 21:00:33 +0200
User-agent: Mozilla/5.0 (Windows NT 6.1; WOW64; rv:60.0) Gecko/20100101 Thunderbird/60.8.0

Am 16.07.2019 um 17:50 schrieb Andrea Berlingieri:
Hello lwip

Let me introduce myself. I'm a CS student at the University of Bologna working 
on the Virtualsquare
project (you can find some infos at 
http://wiki.virtualsquare.org/wiki/index.php/Main_Page, although
the wiki has not been updated yet with the last development efforts. The most 
updated development
versions of the Virtualsquare projects are available on Github, at 
and at http://github.com/rd235). In particular, I'm working on the vuos 
project, the successor to
view-os. Basically the idea of vuos is to implement a partial virtual machine 
by intercepting the
system calls a virtualized process makes and to execute userspace code in their 
place. Vuos is
modular: this means that we can choose which system calls to virutalize and 
which system call will
be directly forwarded to the underlying kernel. As of right now, the project is 
made for Linux and
for the x86_64 architecture (because of the use vuos makes of ptrace to 
intercept the system calls),
and the system call support is not complete yet (some system calls are not 
supported at the moment),
but the project is still under active development.

In particular, I'm currently working on a submodule for the vunet module of 
vuos, which virtualizes
the system calls of the BSD socket API (socket, bind, connect, etc.). This 
submodule uses lwip as
the underlying network stack, instead of using Linux' TCP/IP stack. In the past 
(2006-2011) a fork
of the lwip project was made to add support for IPv6 and some other features. 
Now I'm trying to
write a new submodule based on the latest version of lwip, and in a way that 
makes it possible, for
the two project (lwip and vuos), to develop independently while maintaining 
compatibility (where the
submodule acts as a "bridge" between vuos and lwip).

In vuos both the modules and the submodules are compiled to obtain shared 
objects, which are then
dynamically loaded/unloaded at need. So it is possible, for example, to insert 
the vunet module of
vuos to virtualize the BSD socket API, and then to mount many different 
networks stacks as
submodules. The virtualized processes can then decide which stack to use for 
their networked

The way I'm tying to implement this with lwip is to compile lwip as a shared 
library, through the
Unix port of lwip, and to load it dynamically with dlmopen, which lets me load 
many different
instances of the same dynamic library. This way a process can mount many 
different instances of a
lwip TCP/IP stack, each with its own configuration and without interferences 
between them. Then,
each system call of the socket API is intercepted and handled appropriately 
(some calls are directly
redirected to their lwip counterpart, some require a little more work).

Now, working with lwip to implement the vunet submodule I've encountered some 
problems that I would
like to discuss with you. I also have some expansion proposals, which I would 
also like to discuss.
Here's a list:

  - I've seen that in lwip there is a mechanism to specify some hooks that will 
be called at the
    appropriate moment. Among the currently defined hooks I unfortunately 
couldn't find the one I
    needed. That is, a hook that is called whenever there is an event on a 
lwip_socket. I need it
    because when an event occurs I need to "notify" the vuos core. The first 
modification I made was
    to add such a hook in opt.h, and the appropriate code in event_callback, in 
sockets.c. As of
    right now, the source code for the submodule is placed in a single source 
file, named
    vunetlwip.c. This file also contains the definition of the hook function. 
Now, this initially
    made for a strange arrangement, where vunetlwip.c depended on the headers 
of lwip, and lwip was
    linked against the vunetlwip.so shared object to link the definition of the 
hook function. This,
    unfortunately, didn't even work, as the hook function had references to 
functions of other shared
    objects of the vuos core that couldn't be linked with lwip without making a 
mess. As of right
    now, for testing purposes, the hook function code has been moved in lwip 
(in a separate file) and
    all it does is notify the events on a Unix domain socket, which is listened 
to by a thread
    created by the vunetlwip submodule, which, in turn, notifies the vuos core. 
This is not ideal,
    and I thought that the best solution in this case would be to add a 
mechanism in lwip to specify
    the hook functions at run time, instead of directly defining them at 
compile time. So, my first
    proposal would be to add such a mechanism and a hook for when there are 
events on a socket.

I'm open to adding another hook, but you'll have to do the "specify the
hook functions at runtime" yourself. But I don't see a problem to do so:
just define the hook to be e function that checks if a real callback has
been registered. No need to move that into the actual lwIP code.

  - I've seen that the sockaddr structures have been redefined in lwip. It 
looks although that the
    "model" after which they were made was that of BSD Unix, as each sockaddr 
struct adds a len field
    before the others in the struct. This kind of "breaks" in Linux, as the len 
field is absent, and
    using a sockaddr as defined in Linux with, for example, lwip_connect, 
casuses a mismatch between
    the fields of the two structs, and thus unexpected behavior. This happens 
because the virtualized
    process usually runs a program that assumed Linux as the target operating 
system, and thus uses
    sockaddr as defined in Linux when using bind, connect, sendto, etc. It 
seems though that the len
    field at the begininnig of the struct is non standard and non portable (as 
I could understand
    from this SO answer:
    So, my second proposal would be to modify the sockaddr structs to either 
remove the len field, or
    to at least move it at the end of the structure, so that Linux' and lwip's 
sockaddr agree at
    least on the first part of the struct.

I don't have a problem with adding an option (e.g.
LWIP_SOCKETS_SOCKADDR_HAVE_SALEN). However, you'll probably find more
differences in the header files. Because of that, Joan has been working
on LWIP_SOCKET_EXTERNAL_HEADERS to replace our header with the original
OS header (Hurd in his case).

You might still need to get rid of 'sa_len' then...

  - In the old lwip fork I mentioned above (which, from now on, I will refer to as 
"lwipv6") the
    following features were added: a vde driver, to work with vde (also a 
virtualsquare project,
    https://en.wikipedia.org/wiki/Virtual_Distributed_Ethernet), SLIRP protocol 
support, and NAT
    support. I think they would be some interesting additions to lwip. Let me 
know if you're
    interested in porting them upstream.

Why "lwipv6"?

I'm not interested in porting anything upstream myself right now, but if
you want to send a patch, we could see if it's a match for lwIP that's
good to integrate.

But I'd be certainly interested in a link to the NAT implementation,
just to have a look ;-)

  - I see that lwip is built mainly for embedded devices and one of its most 
prominent features is
    the low memory footprint. While I think this is all good and important, I 
also think that it
    would be nice to be able to choose between the lwip internal memory 
management and the dynamic
    memory management provided by the Linux kernel when compiling the lwip 
shared library. In lwipv6
    code was added to use the glibc's malloc when mem_malloc was called, 
glibc's free when mem_free
    was called, and so on. I think it would be a nice feature for the Unix lwip 
library. The choice
    could be made at compile time through some configuration constants.

That's available since ages: MEM_LIBC_MALLOC to use 'malloc' instead of
'mem_malloc' and MEMP_MEM_MALLOC to use heap allocationi instead of the
memp pools.

  - I've seen that in lwip each netif is associated with a single ipv4 address, 
a single netmask and
    a single gateway. Now, in lwipv6 a modification was made to support lists 
of ip addresses, so
    that a single interface could be associated with more that one ip address. 
Now, introducing this
    feature to lwip would require also a modification to the route function, 
although this wouldn't
    seem to be too complicated, at least conceptually (as, instead of looking 
at a single address, a
    search on a list would be performed). So, my next proposal would be to add 
this feature.

I've written before that I'm very open to this. In fact, our IPv6
support provided this from the beginning. It might just not be easy to
keep all apps and ports compatible to such a change...

  - Stack configuration (setting addresses, setting interfaces, getting 
interfaces infos, etc) is
    made through netlink in my vunetlwip module. Even ioctls are intercepted 
and converted to the
    appropriate netlink counterparts. Now, while working on the address 
information retrieval (the
    GETADDR message of the rtnetlink protocol), I couldn't be able to find the 
prefix len for a given
    ipv6 address. As of right now I simply assume that the prefix len is 64 and 
return that, but that
    is a strong assumption. This is mix between a proposal to add such a field 
somewhere in lwip, and
    a request of help on how to find this information, if it is already encoded 
somewhere I couldn't

Yeah, well, after Ivan left the project, IPv6 got less attention than it
needed. Patches are welcome :-/

  - In lwipv6 a modification was made to support multiple routes for a netif. 
That is, for each netif
    a list of routes was kept. It seems to me though that in lwip routing is 
very simple: packets are
    routed to the interface that has a matching ip address prefix to that of 
the dest ip address (if
    source address routing is not involved). My next proposal would be to add 
such a feature to lwip,
    as I think it may be useful, although it may make routing more complicated.

We have a hook for routing already. Such an improved implementation
could be provided in contrib/addons as a standard addon.

  - Now, this last one is kind of low priority. Basically, while testing with 
ping, I noticed that
    the iputils version of ping for Linux uses a 
socket(AF_INET,SOCK_DGRAM,IPPROTO_ICMP) call, in the
    case of ipv4, to create a socket to send ICMP packets. Now, lwip assumes 
that a socket that uses
    SOCK_DGRAM as a type is a UDP socket. This caused some unexpected behavior. 
As of right now I
    simply intercept the socket call and transform a 
    into a socket(AF_INET,SOCK_RAW,IPPROTO_ICMP) call. This kind of fixes the 
problem, but is not
    ideal. My proposal would be to add support for this case of socket, if a 
configuration constant
    has been defined (it seems to me that this kind of calls are only made and 
supported in Linux. In
    this case the kernel handles the call in a special way, and the created 
socket is only for ICMP
    echo requests: 
https://stackoverflow.com/questions/8290046/icmp-sockets-linux , third answer).
    I'm not too sure about this, maybe we can discuss it a little.

From what you wrote: send a patch and I'm sure we can accept it.

So, this is all I've come up with as of right now. I would like to discuss 
these topics with you.

I'm sorry for the long email. This is my first time posting to this mailing 
list, so If I did
anything wrong let me know, so that it won't happen again. I wish everyone a 
good day.

Well, the mail was too long ;-) At least to read it on the phone...
Other than that: you might have noticed we need more developers, so
please share your code!


Andrea Berlingieri

reply via email to

[Prev in Thread] Current Thread [Next in Thread]