[Date Prev][Date Next][Thread Prev][Thread Next][Date Index][Thread Index]
Re: Re: A more general question about curl
From: |
hyazinthe |
Subject: |
Re: Re: A more general question about curl |
Date: |
Sun, 11 Sep 2022 18:36:11 +0200 |
Hello everyone,
wget2 community is hearing our conversation and is reaching out to us for
collaboration:
story - short:
> On 10. Sep 2022, at 20:53, Tim Rühsen <tim.ruehsen@gmx.de> wrote:
>
> just found this issue: https://gitlab.com/gnuwget/wget2/-/issues/550
>
> Feel free to add the details needed for GNUnet.
story - medium:
> On 10. Sep 2022, at 20:47, Tim Rühsen <tim.ruehsen@gmx.de> wrote:
>
> TL;DR the wget2 project is willing to help and a multi API may be easy
> to implement. But someone has to write down the exact needs.
Full story, its rest -
https://lists.gnu.org/archive/html/wget-dev/2022-09/msg00008.html :
> On 10. Sep 2022, at 20:53, Tim Rühsen <tim.ruehsen@gmx.de> wrote:
> I try to explain what libwget does today and that it seems very straight
> forward to implement an API - and by the way - everybody is invited to do
> that :-)
>
> Libwget has several layers of abstraction of accessing the network stack(s).
>
> So you have indeed the synchronously, super simplified
> response = wget_http_get(...) like in example/http_get.c
>
> Then there is async HTTP API layer with more control over the details, see
> example/http_get2.c.
>
> err = wget_http_open(&conn, url); // comes back immediately
> err = wget_http_send_request(conn, req); // comes back immediately
>
> resp = wget_http_get_response(conn); // waits until error/timeout or response
>
> wget_http_close(&conn);
>
> While you could send several requests over a single connection, HTTP/1.1 has
> issues with it. But it works fine with HTTP/2. In this case
> wget_http_get_response(conn) can be called in a loop, returning the finished
> response in the order they came in. You can also have as many open
> connections as you like - but what is missing, if I understood correctly, is
> an API that fetches the responses from more than one connection at once, like
>
> resp = wget_http_get_response(conn1, conn1, ..., NULL);
>
> Now there is also a TCP+SSL API (used by the above mentioned high level
> functions). This API is works asynchronously. It is like
>
> tcp = wget_tcp_init()
> ... set all kind of configurations to the 'tcp' handle ...
> err = int wget_tcp_connect(tcp, host, port) // returns immediately
> wget_tcp_write() // returns after timeout or immediately if no timeout
> wget_tcp_read() // returns after timeout or immediately if no timeout
> wget_tcp_close()
> wget_tcp_deinit()
>
> Internally, this API uses select/poll, but just uses a single 'tcp' handle.
>
> Now, what a "multi" API basically would look like is e.g.
>
> a function wget_tcp_select(array of tcp handles, timeout) which can be called
> in a loop and which returns an array of "ready" tcp handles (ready for write
> or read, configurable per tcp handle).
>
>
> For me it looks like this is straight forward to implement (depends on the
> details / requirements).
>
>
> Internally, wget_ready_2_transfer(int fd, int timeout, int mode) in
> io.c just needs a companion function that takes a list/array of fds.
>
> If there really is interest from the GNUnet community, why not open an issue
> at https://gitlab.com/gnuwget/wget2/issues to discuss the details and the
> needs. Once we agree upon the details, the implementation can be done by
> anyone - whoever likes to pick it up.
Greetings,
Bastian Schmidt