Modern Lightweight channEl Service

Wuhuu, latest MlesTalk beta has been promoted to stable! Check https://mles.io/blog.html for details. Happy talking!

MlesTalk FOSS beta has been released for testing and feedback. Please check https://mles.io/blog.html for more info. Happy talking!

A unique concept of Mles is its message history where some of the previous messages are saven in-memory by the Mles server and sent to a client after connection establishment. This allows clients to be implemented without any persistent storage as long as they can handle message duplicates during reconnects.

With peer networking, the message history handling gets trickier as downstream peer nodes may flush upstream higher hierarchy nodes' message history by sending their own history during reconnection. Thus, with Mles v1 the history is not sent upstream at all. This may mean lost messages between clients.

With Mles v2 we can improve the design by having message sequence numbering between peers. Combined with unique nodeid, the receiving peer can identify already seen and forwarded messages and not forward duplicates if the history is sent after reconnect. The mechanism can be easily added to peer message headers. It will save a lot of processing in the network as a whole and avoid lost messages. Nice!

The plan is the sequence numbering will be introduced together with the peering feature. If you have ideas how to improve it even further, please do comment. Cheers!

After some proof-of-concept, peering does indeed work fine with the additional peer-field. However, it does not allow a way to multiplex messages over a peer connection. I think we want that for peering.

To do that, a better approach is probably to introduce a new peer-specific message for the handshake, e.g. { "peer-id": x, "auth":"optional hash" }. To leverage the same connection for several uids, the peer connection will always encapsulate the channel header with the message to be able to identify different uids. This is some overhead, but still better than having connection per channel between peers. This is quite similar to Mles v1.

For simplicity, bundling messages over peer connection will not be supported at this point.

As always, comments are welcome!

As Mles is a hierarchical protocol that allows the definition of at least one peer, peering is a very relevant feature. With Mles v2 the peer node concept was postponed to be defined in detail later. And here we are!

The peering concept itself is similar to Mles v1, an Mles node can define a peer that points to a higher hierarchy node in the Mles network. In v2 the peers will do a simple handshake. This ensures that peer connections are identified and optionally authenticated both ways. Also, loop avoidance is built into the protocol naturally. When a node connects to a peer, it will send roughly the following message

{
	"uid":"randomly generated for the node",
	"channel":"example",
	"auth":"hash",
        "peer": true
}

The receiver will reply to the message its uid message which completes the handshake. The initial peer message or its reply will not be forwarded to other clients. However, they are forwarded to other peers. After this, the client messages will be multiplexed over this peer connection.

If there is a valid peer connection in a node, the node does not forward message history to clients, the higher hierarchy peer will do it when a new client joins a channel. This is similar to Mles v1. The peer uid will be saved to uid-table and prevent accidental loops in the peering. Thus, it is safe to connect nodes bidirectionally together or into a ring: the network will automatically organize into a loop-free hierarchy.

So, this is the future peering concept in general (to-be-implemented). Nevertheless, to further improve the peering availability and make it more difficult to monitor which channel members communicate in the network by following packets-only, it would be preferable to be able to define several peers, even by a file or via http get from a location. With several peers defined, there are choices to be made on how to organize the peering when network issues occur. Comments are welcome on how to do it best!

And of course, comments in general to peering, especially if you catch an issue in the design, are very welcome as well.

Mles v2 is now published! Hurray!

It ended up to be pretty amazing upgrade and looks great now. Feedback is still welcome, of course!

Thanks! Happy and safe messaging!

In the previous post, we saw how to write a simple websocat client for Mles. But how to run the server itself?

To run a server, you'll need to follow below steps: - Acquire a public Internet server with a static IP + domain - Open TLS port 443 of firewall - Ensure that your wwwroot-directory has the static web pages under it e.g. static/your.domain - Run Mles server as root (due to port 443) with Let's Encrypt caching and debug logging enabled as shown below

RUST_LOG=debug mles --domains your.domain --cache . --wwwroot static

You can have several domains listed e.g. --domains your.domain --domains www.your.domain.

The full usage:

Usage: mles [OPTIONS] --domains  --wwwroot 

Options:
  -d, --domains   Domain(s)
  -e, --email       Contact info
  -c, --cache       Cache directory
  -l, --limit       History limit [default: 200]
  -w, --wwwroot   Www-root directory for domain(s) (e.g. /path/static where domain mles.io goes to static/mles.io)
  -s, --staging            Use Let's Encrypt staging environment (see https://letsencrypt.org/docs/staging-environment/)
  -p, --port         [default: 443]
  -h, --help               Print help

It is out already in crates.io mles v2.0.0, let me know how it works for you. Enjoy!

As Mles 1.0 had a strong preference to implement authentication of a client, such a mechanism will be part of Mles v2 as well with an optional auth-field.

{
	"uid":"",
	"channel":"",
	"auth":""
}

When MLES_KEY environment variable is set in the server, the auth field will be validated against SipHash over uid, channel + MLES_KEY. The auth feature will be implemented post-v2 release in case requested. siph crate can be used by clients to generate valid auth-fields.

Peering works as before on the logical protocol layer, the peer server does not need to differentiate a real client from a peer. The peering server will not send message history to new connections if it has an active connection to its peer server. The peering feature as well will be implemented post-v2 release in case requested.

An example client session between Alice and Bob with websocat looks like this:

% websocat wss://mles.io --header "Sec-WebSocket-Protocol: mles-websocket"
{ "uid":"alice", "channel":"example" }
Hello Bob!
{ "uid":"bob", "channel":"example" }
Hello Alice!

Mles protocol version 1.0 is deprecated and will be obsolete from the beginning of the year 2024 in favor of Mles v2.

If you have implemented Mles 1.0 protocol-based things on top of mles-websocket protocol, the upgrade is very simple: just send as a first message to the Mles-server a JSON structure based on the Mles v2 header (the JSON structure will be finalized soon). No other changes are needed!

If you have implemented Mles 1.0 based protocol, please consider upgrading to Mles v2. The benefits are:

• up-to-date simplified server framework with even better performance
• safety built-in with mandatory TLS messages via Let's encrypt-certificate over ALPN-TLS
• only port 443 needs to be open, no need for the 80 or 8077 ports anymore

Comments are welcome! Cheers!

Previous prototype round leaned towards TLS-only solution and the direction has been fruitful. It offers a possibility to simplify Mles v2 further.

It has been possible to connect to Mles server via WebSocket proxy. This is handy as e.g. JavaScript web browsers support this out-of-the-box. There is also nice Rust tooling around that via e.g. websocat which supports secure WebSockets (WSS). As can be imagined, also JSON is supported by JavaScript and other tools well.

With a JSON structure as a connect initiator over WSS, we can simplify Mles v2 protocol to work over WSS-only. The SipHashing defined earlier for Mles v2 can be moved to be an internal operation which guarantees the same duplicate user identification and peering as before. Connections will be guaranteed to be secure as WSS to public 443 port guarantees proper certificate checks.

The above has been prototyped already and works splendidly well. The coming weeks are used to finalize the implementation to production quality. Earlier Mles v1 will be deprecated as of now. IANA port 8077 reservation can also be dropped in the future.

In practice, the Mles web-proxy and server will be merged to be a server binary. The client and library implementations are not needed anymore. A simplified client with websocat command line will be provided as an example. This all helps in maintenance of v2 for the next decade. Peering-feature will be postponed from first release.

As an attempt to break away from big company ecosystems, the MlesTalk application will be targeted to be published later as a new MlesTalk FOSS with strong E2EE support based on the Zpinc protocol implementation.

Further updates and guidance how to help will be provided in later updates here.

Thus, big changes are coming, any comments or questions so far?

After some weeks of prototyping, what are the results?

Libp2p as such was working as specified. Nevertheless, there are couple of issues regarding the Mles protocol:

1. frame check works only for the frames we receive on the channel

This means that to be able to check the validity of the first frame, the server would need to be on the same channel as the clients. This is problematic, as the channel should be selected by client to be anything...

OK, let's assume that we pick reserved "__mlesv2" channel for the first frames. It looks hacky already, but anyway, what would work or not? And we come to the second issue:

2. retransmit during reconnection does not work per channel

For new clients, we cannot really select any connection to send to as there no existing api to publish only to a peer. The server would need to flood the network to be able to send the message history to a new client. And if we have the "__mlesv2" way of communication, all clients of any channel would receive them. Not a way to go.

Well, that's what prototyping is for. We quickly spot possible issues with the design.

When I think about it, perhaps a simple TLSv1.3 connection would be fine even for direct server connection. There is some hassle with the certificates, but it may be possible to use the TLS-ALPN-01 way for public server and manual methods otherwise.

If no one comes up with a better idea, this is in the focus of next prototyping effort and very likely the first alpha implementation for Mles v2.

Comments are welcome! You can also comment if you think this is the way to go!

With Linux based devices, Mles 2.0 support is easy to achieve. There are a lot of existing libraries and programs that allow to create WebSockets over TLS connection with the Mles 2.0 simplified Json header structure. ChatGPT can write with little guidance a basic bash program that communicates over updated Mles.

How about embedded devices without Linux? Or Rust no-std programs? In the Rust ecosystem there exists already support for both, embedded websockets and embedded TLS. They have no-std ability. I already have an ESP32-Rust-C3-devkit based device where I can prototype this and share the implementation.

Summa summarum, it should be possible to use updated Mles 2.0 even with embedded devices without Linux. It is important as the new Mles 2.0 APIs have guaranteed security built-in, TLS for the websockets API and Noise for the distribution API. These APIs should provide a valid baseline to build on long into the future.

We have in previous posts dvelved into protocol and messaging framework improvements. The ideas presented earlier seem still solid after some time. Grand.

How about the server internal structure? It has two APIs, WebSocket interface via TLS port, and port 8077 for peer-to-peer server connection. Earlier, the websocket-proxy was a separate instance and the server itself had either centralized or peer configuration. With the new full peer-to-peering capabilities, the same logic can be applied or the instances can be merged. Running port 443 open is not everyone's cup of tea, so if the capabilities are merged for simplification, some configuration options are needed to define the proper roles for the instance. Comments are welcome!

Regarding the Mles 2.0 library support, the proper improvement would be to still offer libraries for clients with no-std support (now missing). The server part will leverage the libraries, of course.

The next steps are to prototype the implementation into port 8078 and see how it works. Initial development already exists. Please comment, or let me know otherwise, if you'd like to join either in development or testing.

Cheers!

Mles v1.0 has a hierarchical structure with a centralized-looking main server. It is in theory possible to have a ring of servers but that is not available anywhere at the moment and the such environment might become hard to maintain.

More optimal would be just being allowed to use peer-to-peering. The relevant features would be similar to 1.0 still, guaranteed message delivery for the message history and scalable publish and subscribe for the selected shared channel (i.e. topic) in a secure way. With the rise of crypto-blockchains, such peer-to-peer libraries are nowadays available, widely used, tested, and maintained, such as libp2p. As a protocol e.g. gossipsub with noise + yamux would be fairly good from the requirements perspective. Veilid is also a newer option, but is less documented at the moment.

In the end, the library implementations and their APIs matter the most. Libp2p in Rust allows to check messages received in their API and this would allow to check the SipHash checksum/authentication in the connection init.

The Mles v2.0 server itself would then listen to client connections and connect to bootstrap server (list) during startup. A good question is how would the clients connect then? They could indeed connect via a selected libp2p connection to port 8077 or via wss to port 443. A non-secure connection would not be allowed anymore.

Would this be then simple for clients to implement? I think so. Clients could be easily written with libp2p API or a netcat-like simple implementation could be created with websocat. A prototyping is needed to finish the details, of course. That's enough for today. In case you end up prototyping the above things, let me know how it worked!

Mles originates from October 2016 and the Mles message structure has been the following since the v1.0 release

    0                   1                   2                   3
    0 1 2 3 4 5 6 7 8 9 0 1 2 3 4 5 6 7 8 9 0 1 2 3 4 5 6 7 8 9 0 1
   +-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+
   | ASCII 'M'(77) |            Encapsulated data length           |
   +-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+
   |            CID (lowest 4 bytes of initial SipHash)            |
   +-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+
   |                                                               |
   +                          SipHash                              +
   |                                                               |
   +-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+
   |         CBOR encapsulated data (uid, channel, message)        |
   +-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+

It had pretty modern aspects included such as

• Large data length support, up to 16 MB framing
• SipHash authentication
• CID handling with peer servers
• CBOR

Even though it has served well for almost a decade now, there exist certain issues with it as an open Internet protocol header, such as a) Framing does not have a checksum b) The default SipHash authentication does not work really for connections behind NAT c) CBOR d) Header overhead

Let's check the above cases in detail.

a) Why would we need a checksum over TCP you may ask? Well, you do not need it for data for sure, TCP has one already. Nevertheless, the server side has use for it to identify and drop false random-data bot-connections. And there are often a lot of them, which just happen to start with ASCII 77.

b) Yes, the SipHash did a good job authenticating the TCP endpoints. Unfortunately, behind a NAT the client cannot really know its IP, so shared key was in the end the only working solution for IPv4 servers.

c) CBOR as an (IETF) format sounds nice, but in practice is awkward to both implement and use. It does not really save that much space compared to e.g. JSON, it cannot be easily read on the captures and the most popular crate is now deprecated.

d) Header overhead is pretty extensive for very short messages. The specification says that the use of SipHash can be ignored, but still needs the header fields and CBOR structures for every frame. Do we really need them, every time?

You may have encountered or noticed other issues too regarding the message structure. If so, please let me know, let's fix it.

Regarding the above issues, what would be the concept for the next-generation message structure then? Perhaps not that different, to be honest, see below draft:

    0                   1                   2                   3
    0 1 2 3 4 5 6 7 8 9 0 1 2 3 4 5 6 7 8 9 0 1 2 3 4 5 6 7 8 9 0 1
   +-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+
   | ASCII 'N'(78) |                  Data length                  |
   +-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+
   |                                                               |
   +                          SipHash                              +
   |                                                               |
   +-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+
   |           JSON encapsulated id data (uid, channel)            |
   +-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+

Even though the fields are quite the same, the way we use them will be different and hopefully will solve all of the above issues:

a) The SipHash field will include the header as well and therefore be the checksum for it. The weak dependency on CID will be removed, obviously, as there is no CID anymore, the SipHash is the CID.

b) The SipHash authentication for IP endpoints will be dropped. A shared key can still be part of SipHash authentication, by default the header + the channel are the authentication.

c) Let's drop CBOR and use JSON. And especially, let's not encapsulate the data into JSON. JSON is supported everywhere and is easy for client implementation.

d) After the first initial frame, the connection is authenticated and identified, we do not need the SipHash or JSON anymore. The next frames will use the following format:

   +-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+
   |                       Data (raw bytes) ...                    |
   +-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+

The first frame cannot hold more data than the JSON identifiers, but after that we avoid all the overhead allowing the application to decide the framing.

That's it for this time! This is all draft and you are free to comment, thanks in advance! The presented changes will certainly cause changes to other areas of the protocol. What they are, we check in detail next time. See you then!

Awesome new networking framework, reminds me of Mles design principles.

Mles is aging and references to "modern" and "lightweight" need constant evolving. The earlier state-of-the-art features need an upgrade.

So, what would be the preferable new state-of-the-art technology for next generation?

Based on earlier feedback for Mles during its existence, a more distributed approach would be preferable (even though it is not quite centralized, at least in a common sense). Another suggested feature is strong end-to-end encryption with hidden group member interactions.

Not bad suggestions. And they are nowadays achievable with surprisingly low effort.

Personally I think also CBOR encoding has proven to be awkward, why not ditch it for JSON headers to ease client implementation development? It would be awesome to be able to write a client with a few lines of bash + Netcat. I would call that "lightweight" :)

As a list, I'd summarize the next gen as follows:

• Fully peer-to-peer communication
• State-of-the-art encryption with established distribution protocol (e.g. Noise)
• Hidden protocol headers to avoid group level user interaction monitoring
• JSON format for optional extended headers
• Anything as payload

I already have a prototype implementation in my mind. I'll publish a 2.0 beta at some point. Let me know if something is missing.

Welcome to the federated Mles channel!