Experimental Software

This is still very much early-days, but I'm thinking about building a distributed link aggregator. Some notes:

• no servers (all P2P) except to connect peers
• no "instances" so everything is in one namespace (so like reddit /r/community, not lemmy instance/c/community)
• everyone is a moderator - moderation based on a web of trust type system
• desktop first, mobile/web later - web would require servers, mobile may have platform-specific issues
• no plans to integrate with fediverse, but could be possible
• initial intent is to be text-only - file storage could be supported later in an opt-in basis (for now, just link to an external img host), but I'm worried about disk storage since there's no central data lake and everyone needs to chip in some storage space

Some technical details:

• written in Rust and ReactJS - Tauri-based
• uses Iroh for data synchronization
• looking into Appleseed and maybe TrustNet for moderation

Current status:

• rewrote Appleseed and part of TrustNet to Rust for perf testing last weekend (Rust is waaay faster) - will probably rewrite again once I finish reading the paper (current code is a direct port)
• have basic Tauri+Iroh app, but it's a glorified chat app w/ no syncing between peers (a peer can join a chat, but will lose access if the host goes down)
• reading source code for SimpleX (re-learning Haskell too) - could be interesting for DMs or something

I'm not sure how long I'll work on this, but I'll probably release some libraries under a relatively liberal license, so something like LGPL at the most stringent, but probably Apache/MIT (depends on if I need to clean-room the appleseed implementation).

So, if there's interest here, I'm happy to post updates for discussion periodically.

I been looking at ways to compress data so it can be distributed easily by making it as small as possible as long as the random number generator is consistent through all devices.

It will be computationally expensive to compress but not as expensive to decompress.

What I do is I take all the text/bytes of a file and then create a (char|byte)sheet that contains each unique character that is in the file.

After that, I start at 0 and increment by 1 for the seed. I use Python's random library and seed it with the int. I use random.choices((char|byte)sheet, k=length). For each seed, I have two paremeters. A maximum and a minimum for the length of generated chars/bytes.

Uhhh....

Now that I'm writing this I'm starting to see where some of this is falling apart. I'll post the code for now I got some rewriting to do! For now, here's the progress I had!

main.py

worker.py

Our first post, what's the status?