Re: unsuitable protocols and standards that block innovation

[Martin Schanzenbach]

Note that a large variety of standard transport protocols that gnunet
runs over is what we need, not (only) the most efficient and newest
protocol that can transport our payload.
Eventually, peers may want to "hide" inside HTTPS, DNS, Bluetooth, WiFi
etc, ideally with traffic that looks like those protocols.
Otherwise, we would just run over our own transport protocol or TCP
only.

So IF XMPP is widely used and adopted, and is reasonably performant, it
IS a prime candidate for a gnunet transport.


my 2 ct
BR

On Mon, 2024-03-11 at 17:19 +0100, carlo von lynX wrote:
> On Sat, Mar 09, 2024 at 10:30:07AM +0200, email@msavoritias.me wrote:
> > XMPP can work fine through p2p. Its just not being done a lot
> > because of
> > other reasons i can expand if needed.
> 
> Yes, you can rewrite any protocol radically enough to cook rice
> or iron your shirts but then you are really just keeping the
> unfavourable syntax and nothing truly useful.
> 
> Here Google is promoting a JSON API that has all its semantics
> turned upside down from the original XMPP RFC, but they still
> call it xmpp, as it probably sells better and gives an impression
> of legitimacy:
> 
> https://firebase.google.com/docs/cloud-messaging/xmpp-server-ref?hl=en
> 
> What stuck out quickly to me was the presence of a message_id and
> a message_type. Two things that were vehemently refused by the XMPP
> Council 20 years ago when we suggested they should use something
> like this. Message ids got a clumsy extension XML namespace and
> message types were never introduced at all, forcing all applications
> to deduce the semantics of a message by parsing the entire tree,
> even if it wasn't intended for them.
> 
> Ten years earlier Google kicked XMPP out of the Android backend
> saying it is too verbose for mobile. 20 years earlier some people
> probably involved with Google were in favour of ratifying the XMPP
> RFCs even though it had no scalable one-to-many distribution
> strategy. Back then Google Talk became the largest XMPP node
> because it was the only one that was able to do one-to-many
> distribution all inside its proto-cloud-architecture.
> 
> In other words, even back then XMPP was working best when it said
> so on the wrapping but it wasn't actually being used. Same with
> Androids xmppService, and again with Firebase.
> 
> So if we're going to do an efficient new communications system
> over GNUnet, there isn't anything but the #XMPP hashtag that
> could be of any use. Just call it XMPP, but please don't use it.
> No XML, no XEPs and nothing of what's in the RFCs. Just use
> the things that are being used to de-facto instead, and maybe
> consider what it would be like to design a protocol with the
> right approach from the start, rather than this thing which
> started on two wrong feet back in 1998 and never got much better.
> 
> > > Also I doubt the general public would be interested in a
> > > messenger that can't deliver voice, video and stickers
> > > embedded into the protocol natively without overhead.
> > 
> > What do you mean "overhead" or "natively" here? Because every
> > single
> > protocol out there works with extensions. In the sense of virtually
> > all
> > protocols nowadays use webrtc which is not native to anything
> > (except web i
> > guess but what is web :P )
> 
> That's why we still don't have a real-time capable walkie-talkie
> type of communication tool. The latency introduced by launching
> external protocols rather than using existing streams immediately
> makes that impossible. You can send voicies and you can negotiate
> phone calls, but neither of these are as comfortable to use as a
> walkie-talkie.
> 
> > and second can you give an example of stickers integrated natively
> > into the
> > protocol? Because I can't imagine why stickers would be a protocol
> > level
> > thing since they are just pictures. Its al UI. The only thing a
> > protocol has
> > to dictate is how to share packs which xmpp already does.
> 
> When someone is low on data, downloading a whole sticker pack just to
> be able to show one of them is expensive. That's why Telegram
> sometimes
> just sends you one sticker without passing the entire pack on. Also
> having to go fetch the graphics instead of receiving them immediately
> introduces waiting time - you get these grey boxes announcing that 
> something will show soon, but you have to wait until it actually
> loaded.
> Not saying that this specific example is better to solve differently,
> but it has drawbacks to consider. In order to not transmit the same 
> picture several times you either must know what the recipient already
> has, or you end up using a pull protocol with its latency. How is
> latency
> still a big deal these days? The planet hasn't gotten smaller and
> onion
> routing is still on the list of things we'd like to achieve, so
> having
> messaging systems less prone to latency is good.
> 
> > > IMHO it makes more sense to adapt the open source Telegram[2]
> > > implementations to connect to a localhost server which then
> > > offers backend functionality for Telegram to run over GNUnet.
> > 
> > I agree that telegram has a good UI but i fail to understand what
> > that has
> > to do with protocols. Unless you mean to also get the telegram
> > server and
> > change the whole thing to work over gnunet. Which at that point:
> 
> Since the protocols of XMPP aren't recommendable at all, Telegram's
> JSON APIs are less bad and easier to adapt. Of course we could also
> do an even better messaging syntax and protocol. We've pushed the
> envelope times before.
> 
> > 1. If you are running a seperate process to connect over localhost
> > is it
> > really p2p?
> 
> Having the GNUnet stack in a single process is on the todo list for
> GSoC.
> A Telegram-adapter would be yet another GNUnet service out of many.
> As long as it is localhost, the whole architecture is still P2P.
> 
> > 2. I wonder how much work it would take to convert it to GNS and
> > gnunet
> > because i bet there a whole lot of assumptions in there about DNS
> > and
> > servers and such. So maybe the effort wouldn't be worth it.
> > Especially with
> > how easy android or Gnome make it to have a beautiful UI (if the
> > app is
> > native).
> 
> I would guess Telegram clients to be a lot more straightforward than
> XMPP ones, as they don't use DNS at all (they just connect to the
> cloud).
> So this point of critique applies to XMPP software much more than to
> a
> Telegram client.
> 
> > > [1]https://about.psyc.eu/XMPP#Technical_Problems_with_XMPP
> > > [2]https://telegram.org
> 
> > Regarding your technical problems article some remarks:
> > 
> > - XMPP can have framing if done over websockets. So thats not a
> > compromise
> > you have to make anymore if you want framing.
> 
> So you aren't actually using RFC-compliant XMPP. That's ok.
> 
> > - XMPP can work over Radio with 75 bits/s and minutes of
> > reliability. Its
> > not the most efficient that exists but if you have less bandwidth
> > than radio
> > you can use something else i guess :)
> 
> Did you look at the benchmarks? Why start with an extremely
> inefficient
> syntax?
> 
> > - Streaming parsers exist
> 
> Yes, they needlessly consume CPU to guess what might be coming next.
> Check the benchmarks.
> 
> > - For binary transfer I am looking into Eris or EXI personally
> 
> Because using the protocol you are using is not good enough?
> As if binary data transfer was a separate problem.
> 
> > - The huge rooms problem is actually about presence not anything to
> > do with
> > the protocol. There are two solutions to this:
> 
> IT HAS EVERYTHING TO DO WITH THE PROTOCOL.
> If the RFC forbids any decent distribution strategy, the
> protocol impedes implementing scalable systems unless you
> run them inside a cloud, bypassing XMPP.
> 
> > 1. Disable presence in rooms above a couple thousand people
> 
> That doesn't solve distribution at all.
> 
> > 2. Change to MIX the new way to do group chats in xmpp that is more
> > scalable.
> 
> This implements (1) from what I can see, and thus only reduces the
> amount
> of data distributed a bit, but not the challenge of distributing
> stuff to millions of recipients in a Twitter-like use case.
> 
> > - The multicasting part looks very interesting. Can you expand on
> > how that
> > would work in a distributed network like gnunet? Because it
> > specifically
> > mentions a server solution which we cannot afford here:
> > 
> > > In PSYC <https://about.psyc.eu/PSYC>, by contrast, the server
> > > will contact
> > a set of servers, which will forward the news to another set of
> > servers
> > each, until all the recipients receive the news
> 
> A multicast routing layer has been in the plans for GNUnet since
> before
> we first came asking, which was around 2009. Research has been made
> on
> how to do this, but there is no implementation as yet.
> 
> In GNUnet, nodes would efficiently set up distribution trees to get
> all
> data to everyone who needs it. If done properly it can handle
> Twitter-
> and even television-like use cases with millions watching the same
> video
> stream. The current status quo is that these distribution trees are
> implemented within the cloud, thus you only get to enjoy their
> strengths
> if you surrender your cookies to them and join the cloud hegemoths.
> 
> XMPP was never a challenge to the cloud platforms, because it never
> solved the one thing that needs to be solved in order to scale
> outside
> the cloud: distribution trees. And the way it fails is subtle to
> diagnose, because people who try it simply stop using it, so it
> always
> looks like it's an adoption problem. Humans hate science, so they
> hate
> scientific evidence that tells them that XMPP could never work the
> way
> it got its shoelaces tied back in 1999. Humans love to think that the
> problem is something else. Adoption and extension protocols for
> example.
> 
> > - for an up to date xmpp server count you can see here
> > https://xmppnetwork.goodbytes.im/webgl.html :)
> 
> Of course people can use it anyway, but it just will never be able to
> work for the entire human society the way it is designed.
> 
> > To keep this short your document there seems to be more than 15(!)
> > years
> > ago. So please link up to date versions in the future because XMPP
> > its still
> > being updated. Also lot of the links are dead too.
> 
> The core problem never got fixed.
>

signature.asc
Description: This is a digitally signed message part