marsuplane

Wuhuu, latest MlesTalk beta has been promoted to stable! Check https://mles.io/blog.html for details. Happy talking!

MlesTalk FOSS beta has been released for testing and feedback. Please check https://mles.io/blog.html for more info. Happy talking!

A unique concept of Mles is its message history where some of the previous messages are saven in-memory by the Mles server and sent to a client after connection establishment. This allows clients to be implemented without any persistent storage as long as they can handle message duplicates during reconnects.

With peer networking, the message history handling gets trickier as downstream peer nodes may flush upstream higher hierarchy nodes' message history by sending their own history during reconnection. Thus, with Mles v1 the history is not sent upstream at all. This may mean lost messages between clients.

With Mles v2 we can improve the design by having message sequence numbering between peers. Combined with unique nodeid, the receiving peer can identify already seen and forwarded messages and not forward duplicates if the history is sent after reconnect. The mechanism can be easily added to peer message headers. It will save a lot of processing in the network as a whole and avoid lost messages. Nice!

The plan is the sequence numbering will be introduced together with the peering feature. If you have ideas how to improve it even further, please do comment. Cheers!

After some proof-of-concept, peering does indeed work fine with the additional peer-field. However, it does not allow a way to multiplex messages over a peer connection. I think we want that for peering.

To do that, a better approach is probably to introduce a new peer-specific message for the handshake, e.g. { "peer-id": x, "auth":"optional hash" }. To leverage the same connection for several uids, the peer connection will always encapsulate the channel header with the message to be able to identify different uids. This is some overhead, but still better than having connection per channel between peers. This is quite similar to Mles v1.

For simplicity, bundling messages over peer connection will not be supported at this point.

As always, comments are welcome!

As Mles is a hierarchical protocol that allows the definition of at least one peer, peering is a very relevant feature. With Mles v2 the peer node concept was postponed to be defined in detail later. And here we are!

The peering concept itself is similar to Mles v1, an Mles node can define a peer that points to a higher hierarchy node in the Mles network. In v2 the peers will do a simple handshake. This ensures that peer connections are identified and optionally authenticated both ways. Also, loop avoidance is built into the protocol naturally. When a node connects to a peer, it will send roughly the following message

{
	"uid":"randomly generated for the node",
	"channel":"example",
	"auth":"hash",
        "peer": true
}

The receiver will reply to the message its uid message which completes the handshake. The initial peer message or its reply will not be forwarded to other clients. However, they are forwarded to other peers. After this, the client messages will be multiplexed over this peer connection.

If there is a valid peer connection in a node, the node does not forward message history to clients, the higher hierarchy peer will do it when a new client joins a channel. This is similar to Mles v1. The peer uid will be saved to uid-table and prevent accidental loops in the peering. Thus, it is safe to connect nodes bidirectionally together or into a ring: the network will automatically organize into a loop-free hierarchy.

So, this is the future peering concept in general (to-be-implemented). Nevertheless, to further improve the peering availability and make it more difficult to monitor which channel members communicate in the network by following packets-only, it would be preferable to be able to define several peers, even by a file or via http get from a location. With several peers defined, there are choices to be made on how to organize the peering when network issues occur. Comments are welcome on how to do it best!

And of course, comments in general to peering, especially if you catch an issue in the design, are very welcome as well.

Mles v2 is now published! Hurray!

It ended up to be pretty amazing upgrade and looks great now. Feedback is still welcome, of course!

Thanks! Happy and safe messaging!

In the previous post, we saw how to write a simple websocat client for Mles. But how to run the server itself?

To run a server, you'll need to follow below steps: - Acquire a public Internet server with a static IP + domain - Open TLS port 443 of firewall - Ensure that your wwwroot-directory has the static web pages under it e.g. static/your.domain - Run Mles server as root (due to port 443) with Let's Encrypt caching and debug logging enabled as shown below

RUST_LOG=debug mles --domains your.domain --cache . --wwwroot static

You can have several domains listed e.g. --domains your.domain --domains www.your.domain.

The full usage:

Usage: mles [OPTIONS] --domains  --wwwroot 

Options:
  -d, --domains   Domain(s)
  -e, --email       Contact info
  -c, --cache       Cache directory
  -l, --limit       History limit [default: 200]
  -w, --wwwroot   Www-root directory for domain(s) (e.g. /path/static where domain mles.io goes to static/mles.io)
  -s, --staging            Use Let's Encrypt staging environment (see https://letsencrypt.org/docs/staging-environment/)
  -p, --port         [default: 443]
  -h, --help               Print help

It is out already in crates.io mles v2.0.0, let me know how it works for you. Enjoy!

As Mles 1.0 had a strong preference to implement authentication of a client, such a mechanism will be part of Mles v2 as well with an optional auth-field.

{
	"uid":"",
	"channel":"",
	"auth":""
}

When MLES_KEY environment variable is set in the server, the auth field will be validated against SipHash over uid, channel + MLES_KEY. The auth feature will be implemented post-v2 release in case requested. siph crate can be used by clients to generate valid auth-fields.

Peering works as before on the logical protocol layer, the peer server does not need to differentiate a real client from a peer. The peering server will not send message history to new connections if it has an active connection to its peer server. The peering feature as well will be implemented post-v2 release in case requested.

An example client session between Alice and Bob with websocat looks like this:

% websocat wss://mles.io --header "Sec-WebSocket-Protocol: mles-websocket"
{ "uid":"alice", "channel":"example" }
Hello Bob!
{ "uid":"bob", "channel":"example" }
Hello Alice!

Hi! Mles (Modern Lightweight channEl Service) protocol with Rust-based reference implementation is on the path to upgrade to v2. The new version will be simpler and more secure. In case interested how to make the protocol v2 draft even better, please join discussion and comment on [email protected]. Thanks!

Mles protocol version 1.0 is deprecated and will be obsolete from the beginning of the year 2024 in favor of Mles v2.

If you have implemented Mles 1.0 protocol-based things on top of mles-websocket protocol, the upgrade is very simple: just send as a first message to the Mles-server a JSON structure based on the Mles v2 header (the JSON structure will be finalized soon). No other changes are needed!

If you have implemented Mles 1.0 based protocol, please consider upgrading to Mles v2. The benefits are:

• up-to-date simplified server framework with even better performance
• safety built-in with mandatory TLS messages via Let's encrypt-certificate over ALPN-TLS
• only port 443 needs to be open, no need for the 80 or 8077 ports anymore

Comments are welcome! Cheers!

Previous prototype round leaned towards TLS-only solution and the direction has been fruitful. It offers a possibility to simplify Mles v2 further.

It has been possible to connect to Mles server via WebSocket proxy. This is handy as e.g. JavaScript web browsers support this out-of-the-box. There is also nice Rust tooling around that via e.g. websocat which supports secure WebSockets (WSS). As can be imagined, also JSON is supported by JavaScript and other tools well.

With a JSON structure as a connect initiator over WSS, we can simplify Mles v2 protocol to work over WSS-only. The SipHashing defined earlier for Mles v2 can be moved to be an internal operation which guarantees the same duplicate user identification and peering as before. Connections will be guaranteed to be secure as WSS to public 443 port guarantees proper certificate checks.

The above has been prototyped already and works splendidly well. The coming weeks are used to finalize the implementation to production quality. Earlier Mles v1 will be deprecated as of now. IANA port 8077 reservation can also be dropped in the future.

In practice, the Mles web-proxy and server will be merged to be a server binary. The client and library implementations are not needed anymore. A simplified client with websocat command line will be provided as an example. This all helps in maintenance of v2 for the next decade. Peering-feature will be postponed from first release.

As an attempt to break away from big company ecosystems, the MlesTalk application will be targeted to be published later as a new MlesTalk FOSS with strong E2EE support based on the Zpinc protocol implementation.

Further updates and guidance how to help will be provided in later updates here.

Thus, big changes are coming, any comments or questions so far?

After some weeks of prototyping, what are the results?

Libp2p as such was working as specified. Nevertheless, there are couple of issues regarding the Mles protocol:

1. frame check works only for the frames we receive on the channel

This means that to be able to check the validity of the first frame, the server would need to be on the same channel as the clients. This is problematic, as the channel should be selected by client to be anything...

OK, let's assume that we pick reserved "__mlesv2" channel for the first frames. It looks hacky already, but anyway, what would work or not? And we come to the second issue:

2. retransmit during reconnection does not work per channel

For new clients, we cannot really select any connection to send to as there no existing api to publish only to a peer. The server would need to flood the network to be able to send the message history to a new client. And if we have the "__mlesv2" way of communication, all clients of any channel would receive them. Not a way to go.

Well, that's what prototyping is for. We quickly spot possible issues with the design.

When I think about it, perhaps a simple TLSv1.3 connection would be fine even for direct server connection. There is some hassle with the certificates, but it may be possible to use the TLS-ALPN-01 way for public server and manual methods otherwise.

If no one comes up with a better idea, this is in the focus of next prototyping effort and very likely the first alpha implementation for Mles v2.

Comments are welcome! You can also comment if you think this is the way to go!