Mycelium

none of this is built yet. there's a spec at v0.2 and a build guide with 32 checkpoints, and every checkpoint still says panic("todo"). what follows is the design and why it looks like that. the spec calls it Mycl for now; the name isn't settled.

the short version, from the spec:

Mycl is an offline-first synchronisation protocol for structured application data across a user's own devices. It defines a file format, an authenticated message envelope, and a conflict-detection model. It does not define a server, a merge algorithm, or a user interface.

that last sentence is the point. it's a protocol, not a platform.

what it's for

• every device keeps a full local copy. syncing makes things nicer; it isn't needed to read or write.
• devices on the same network find each other directly. a relay is optional and never in charge.
• every message is signed with a key the receiver can check.
• conflicts get reported, not quietly resolved.
• files on disk are readable and diffable with normal tools.

and the things it deliberately won't do, which matter more: no live collaborative editing, no CRDTs, no character-level merging, no automatic conflict resolution, no sharing between different people, no NAT traversal, no accounts.

why no CRDT

this is the decision everything else follows from. detecting a conflict is the protocol's job. deciding what to do about one is the app's job. a notes app and a media player shouldn't resolve the same conflict the same way, and a protocol that merges for them has taken that choice away.

so conflicts are found with version vectors — a count per device, per key. editing locally bumps your own count and nothing else. comparing two of them gives four answers: yours is newer, theirs is newer, they're the same, or they're concurrent. only the last one is interesting, and it raises a flag and hands both versions to the app.

an earlier draft used one counter and last-write-wins, and why that had to go is the clearest way to explain the problem. two devices are offline, both editing version 3, and both produce a version 4. a single counter can't tell you whether one edit came after the other or whether they happened at the same time. so it can't detect a conflict at all — it can only pick a winner and throw the other edit away. the conflict flag could never have been set.

breaking ties on timestamps makes it worse, because devices disagree about the time, so two of them deciding separately might not even agree on the winner.

when an app really does need a default, the winner is whichever device has the lexicographically greatest id. that's arbitrary, but every device works it out the same way without a shared clock, so they can't drift apart permanently. the timestamp stays for display and debugging and nothing depends on it.

hash-chained history was considered too. verifiable history is nice, but it needs old hashes around to tell a real conflict from a descendant of pruned state, and it invents conflicts once history gets truncated.

proving which device you are

each device makes an Ed25519 keypair on first run and never lets the private key leave. its id is derived from the public key, so a receiver can recompute the id from the key it was sent and reject anything that doesn't match. no shared secret, nobody to trust.

an earlier draft used HMAC, and dropping it is the spec arguing with itself and winning. HMAC is symmetric: to check a tag you need the secret that made it, which in a multi-device setup means every device holds every other device's signing key. so any device can forge a message from any other, and the id becomes a claim rather than a fact. the signature only proved the message came from somewhere inside the group, which you already knew.

two more in the same spirit:

• the signature covers the envelope as well as the payload. an earlier draft signed only the payload, which left the key name and the version data unprotected — so something in the middle could point a write at a different record without breaking the signature.
• proving a key owns its id doesn't prove the key belongs to your device. that gap gets closed by pairing out of band, with a QR code or a short code, and messages from unpaired devices are dropped rather than flagged.

the file format

a .mycl file is TOML, with five sections in a fixed order. the spec is careful about which way round that goes:

The format is a restriction of TOML, not a superset: every .mycl file parses as TOML, but most TOML files are not valid .mycl files.

signing needs one exact set of bytes, and TOML doesn't give you that — key order, spacing and number formatting all vary between writers. so there's a defined canonical encoding, and no floats in signed sections. fractions become scaled integers, so playback_rate_milli = 1500 rather than 1.5.

keys can name a chunk of a record, and that's doing real work rather than tidiness: keys are the unit of conflict detection, so two devices editing different chunks of the same thing never collide.

files stay the source of truth, with a SQLite index over them as a cache you can rebuild whenever. the data stays readable with normal tools and there's no database migration path to maintain.

the part that worries me

the Go core has to be written a second time in TypeScript for the pure logic, because that logic needs to run where the core can't — mainly the browser, which can't open sockets or do multicast discovery at all. two implementations of canonical encoding and version-vector comparison is a real risk, and the spec says so plainly: they can drift, and the bugs would only show up between platforms.

the answer is a shared file of test vectors that both test suites read, written before the second implementation exists. conformance is defined by that data, not by either implementation. pass the vectors or you're not conforming.

what finishing would mean

v0.1 isn't v1.0 on purpose:

A version number is a promise to consumers. Until an application developer can embed this software, there are no consumers.

so v0.1 is the protocol working end to end in one language, driven by a CLI, and v1.0 waits until an app can actually embed it. of the acceptance criteria, one matters more than the rest: two devices edit the same key while disconnected, and when they reconnect both report a conflict and neither throws the other's edit away. hitting a latency target is easy. getting concurrency right is the work.

the build order has the same instinct. encryption comes after plain framing works, because encryption on top of broken framing is impossible to debug. libp2p was turned down for replacing the parts the spec exists to define. TLS lost to Noise because there's no certificate authority in a peer-to-peer setup for a certificate to mean anything.

the build guide names its own stopping point, which is the bit i like most:

If you complete through H4, the project has succeeded. That's a working, authenticated, conflict-detecting sync protocol with an automated partition test. Phases I through K make it usable and shareable; they are not what makes it real. Half-finished is also fine.

still open

file transfer is unresolved — a schema names a file the protocol doesn't move yet, and that has to be settled before v1 means anything. also open: the shape of keys, whether pruning old counters needs tombstones, whether the relay needs to understand the protocol or can just forward bytes, and the name.

Apache 2.0.