Encrypted messages that carry QALS — Streams reborn · qalchat/DESIGN.md

qalchat — DESIGN

Encrypted messaging with in-message QALS value transfer over the Qals stack. Created 2026-09-08 by qalcode autonomous session (project: crypto_currency). Implementation: qalchat/qalchat.py (single file; sole dependency beyond stdlib: cryptography 46.0.3). Proof suites: qalchat/test_chat.sh, test_invites.sh, test_files.sh (files + multi-relay + mesh ferry), qalmcp/test_mcp.sh (MCP server — see qalmcp/README.md) — all green 2026-09-09.

User brief: "a messaging app which is also possible to transfer these qals as values. So the messages get sent for free over this network or could move between meshes. Would be good if we could have it encrypted too."

1. Architecture

     alice (CLI)                                    bob (CLI)
  identities/ 0600                               identities/ 0600
  X25519 + ed25519 + loopd secret                X25519 + ed25519 + loopd secret
       │  1. POST /pubkeys/<handle>  (keyserver, TOFU-pin)   ▲
       │  2. POST /envelopes ───────► RELAY :8830 ───────────┘
       │       {from, to, ciphertext, nonce, ts,     │  3. GET /envelopes/<handle>
       │        msg_hash, sig, ctr, pub}             │  4. DELETE /envelopes/<h>/<id>  (ack)
       │                                             │
       │  relay sees ONLY ciphertext.                │
       │  zero fees — plain HTTP, store-and-forward, │
       │  24h TTL, 64 KiB cap.                       │
       ▼                                             ▼
   loopd :8823  ◄── POST /transfer (HMAC by sender, AU$200/day cap) ──►  loopd
   receipts.jsonl ◄── r_… ──► in-chat ⚡ payment message (inside GCM)
       │
       ▼
   qalnet :9000 ◄── qalpipe anchor: Merkle root of conversation → DataAnchor object
                     (verify-convo: recompute root → fetch object → compare)

Components (all in one file):

Message flow (send): derive keys → seal → hash-chain → sign → POST. recv runs the full gauntlet in order: msg_hash → ed25519 signature → AES-GCM open → hash-chain link → recompute chain locally. Any failure ⇒ envelope REJECTED (exit 3) and left un-acked (tampered mail can never decrypt; a genuine redelivery still can).

2. Crypto — choices, and honest limitations

What is done:

Honest limitations (read before trusting this with anything real):

1. NOT forward-secure. The ratchet is HKDF(root, sender, ctr) — deterministic. Compromise of the conversation root (or either X25519 identity key) recovers all past and future messages. This is not the Signal double-ratchet: there is no DH re-keying, no per-message ephemerals, no erased key state. The mission's "forward secrecy per message" is delivered only as key separation (one leaked message key does not reveal others). A stateful hash-ratchet with erasure is the incremental fix; MLS (RFC 9420) with per-epoch epoch secrets is the real one. Both are §6 upgrade work.
2. TOFU keyserver. First contact pins whatever the relay returns; a malicious or coerced relay can substitute keys at first contact (MITM). After pinning, key substitution is loudly rejected. Phase 1 fix SHIPPED 2026-09-09 — QR/invite-link key exchange out-of-band: invite-pinned handles bypass TOFU entirely and the keyserver is demoted to a tamper detector (§9). Still open for invite-less first contacts. Phase 2: anchor identity-key hashes on-chain (agent DIDs, §9 v2).
3. Metadata is visible to the relay: from/to handles, timing, size, counters. Content is sealed; traffic analysis is not attempted.
4. Relay is trusted for availability, not integrity — but not for delivery completeness: a relay can silently drop mail (you can't prove a negative without per-conversation receipts; the hash chain makes tampering detectable, not suppression). The 24h store-and-forward window is a parameter, not a guarantee.
5. Random nonces (96-bit): fine at human chat volumes; would need per-conversation nonce counters (they'd have to be AAD-bound) at machine fan-out scale.
6. Anchoring commits to ciphertext hashes (msg_hash), not to local plaintext transcripts — editing your local history text doesn't (and shouldn't) flip the on-chain verdict; editing anything the root covers does. Anchors are per-view: each party anchors what it has received; a party with incomplete mail anchors an incomplete (but internally consistent) view.
7. One device per handle (the loopd HMAC secret is claimed exactly once). Multi-device = share/rotate identity, or proper MLS leaves in Phase 2.
8. Audit scope: ~800 lines, written in one session, tested against live services but not externally reviewed. Treat as a prototype.

3. Streams lineage (IOTA Channels → qalchat)

IOTA Streams (archived Apr 2024, alpha, Rust) shipped Channels: Announcement (channel start), Keyload (session keys to subscribed public keys), Signed Packet (public payload + signature), Tagged Packet (encrypted payload), over a Tangle transport. qalchat is the same protocol shape, rebuilt for what survived:

4. Free-over-network + mesh movement

Why messages are free: the relay is ordinary HTTP — no gas, no PoW, no storage deposits. Value moves inside the encrypted payload (a payment is just a message whose text says ⚡ … settle id r_…); the ledger movement itself is a loopd paper entry now and an on-chain qal_credit move later — both feeless on Qalnet.

What's implemented today: - --relay flag and QALCHAT_RELAY env — the same conversation (keys, history, handles) works against any relay URL. Proof: test §6 reads the payment via http://localhost:8830 after sending via http://127.0.0.1:8830. - convo_id = qc1:sha256(sorted(handles)) — contains no relay URL. Handles are portable; conversations are relay-agnostic. - Identity = one 0600 JSON file → copy it to another machine on another mesh and the conversation continues.

Phase 1 sketch — relay federation (design only):

   mesh A relay                mesh B relay
   (superlocal:8830)  ◄──────►  (minirig:8830)
        ▲  envelope gossip (POST /federate {envelopes for handles
        │   homed on the peer, sig-verified, TTL preserved})
        │
   home relay = where the handle polls. Others queue-and-forward.

1. Handle homing — a handle declares a home relay in its published key doc ({"x25519_pub":…, "ed25519_pub":…, "home":"https://…"}). Senders resolve handle → home; any relay can accept mail for any handle and gossip it home.
2. Envelope handoff — relays are peers: POST /federate ships envelopes whose recipient is homed on the peer. Envelopes are already self-authenticating (signature + msg_hash + chain), so a relaying relay needs zero trust: it cannot read or forge, only drop (and chains detect tampering, not drops).
3. Sync protocol — clients store msg_hash chain; on reconnect to a different relay they request "everything after hash X" — the chain gives both sides a reconciliation key (what Bitcoin nodes do with headers, at toy scale).
4. No global namespace — handles are per-mesh unless a mesh anchors a name→keyhash mapping on-chain; until then, QR/invite links carry the full key material out-of-band (fixes TOFU too).

5. Payment flow

alice: qalchat pay bob 250 "coffee"
  │
  ├─► loopd POST /agents/by-handle/bob      → to_agent_id (public info)
  ├─► loopd POST /transfer                  ← HMAC-SHA256(alice_secret, body)
  │     {agent_id, to_agent_id, cents:250, memo:"coffee"}
  │     checks: not revoked · cents>0 · balance ≥ · spent+cents ≤ 20000/day cap
  │     moves 250c alice→bob atomically (lock) · appends receipts.jsonl
  │     returns {receipt_id:"r_…"}
  ▼
  build payment payload:
    {"type":"payment",
     "text":"⚡ alice sent you 2.50 B-QALS (redeemable AU$1:1) — settle id r_… — memo: coffee",
     "pay":{"receipt_id","cents","memo","to_agent_id"}}
  ── AES-256-GCM seal → signed envelope → relay → bob
  bob: recv → ⚡ payment message decrypts inside the conversation
       (settlement reference and receipt travel encrypted, like everything else)

Paper-mode honesty: loopd is a JSON-file ledger (Phase 0 genesis importer for the qal_credit Move object). "Redeemable AU$1:1" is the paper-loop convention — there is no reserve behind it today. The transfer receipt maps 1:1 to a future on-chain qal_credit::transfer call.

6. Anchoring a conversation

anchor bob → canonical order (sender, ctr) → leaves msg_hash → Merkle root → canonical blob (no timestamps ⇒ byte-stable) → qalpipe.anchor_file() (sha256 → data_anchor::anchor Move object on qalnet-dev-1 → tx + object id). verify-convo bob → recompute root over the anchored prefix → rewrite blob → qalpipe.verify_file() reads the chain, compares. Local tampering of any committed field ⇒ TAMPERED (tested both ways).

Upgrade path: per-message anchor receipts today are per-conversation; batching many conversations into one Merkle root per hour (as the QALNET design §"External anchor" sketched) makes external notarization effectively free.

7. Upgrade path

1. MLS (RFC 9420) — replace the symmetric ratchet: TreeKEM gives per-epoch forward secrecy + post-compromise security + multi-device/multi-party leaves. The envelope format (sig+ciphertext+chain fields) survives; the key schedule swaps underneath. Group chats become an MLS group per convo.
2. On-chain qal_credit — move loopd's ledger to the Move package: transfer becomes a transaction, the daily cap becomes contract state, receipts become events. loopd stays as the cap/spend policy engine (holds/settle for jobs).
3. Agent DIDs — did:iota-style object per handle anchored on Qalnet: key rotation kills the TOFU problem, revocation lists kill compromised-device ambiguity, and SpendAuthorization VCs (cf. 09_identity_ai/IDENTITY_AND_AI.md §Qal ID) turn "alice may move up to X/day" into a verifiable credential instead of a loopd config row.
4. QR invites — identity file's public half (x25519_pub, ed25519_pub, home relay) as a QR code / qalchat://invite?... link; scanning pins keys out-of-band, deleting the keyserver MITM window (limitation #2).
5. Federation — §4 Phase 1 sketch; then push (not poll) delivery so the 24h window becomes irrelevant for online peers.

8. Test evidence map (test_chat.sh)

9. Invites shipped — TOFU killed by out-of-band key exchange (2026-09-09)

qalchat's weakest link was the TOFU keyserver: whoever answers first contact gets pinned, so a malicious or coerced relay could substitute keys and MITM the whole conversation from message one. Invites delete that window. invite (qalchat/invites.py + CLI wiring) emits a signed, self-certifying URI — qalchat://invite?v=1&h=<handle>&k=<bundle>&r=<relay> — where the bundle is b64url(canonical JSON payload).b64url(ed25519_sig) over "qalchat-invite-v1|handle|x25519_pub|ed25519_pub|created_ts|relay", rendered as a QR (PNG via qrcode+pillow, terminal half-blocks, or a copyable URI block). accept <uri|png> verifies the signature and version, sanity-checks the timestamp (future ⇒ refuse, >30 days ⇒ warn), and pins the key into data/pinned/<me>/<peer>.json (0600, per-owner because several identities can live on one machine) before any keyserver contact. Thereafter TOFU is bypassed for that handle: the pin is authoritative, and the keyserver is demoted from trust source to tamper detector — every key resolution cross-checks it once per process, and a DIFFERENT key for a pinned handle is the relay-substitution attack itself: the client refuses to encrypt (nothing leaks to the attacker's key) and refuses delivery (envelopes stay un-acked), exiting with a dedicated code. Exit codes now: 0 ok · 1 error/bad invite · 2 verify-convo fail · 3 tampered envelope · 4 KEY SUBSTITUTION. The signature alone does not prove the handle↔key binding (an impostor can sign their own bundle claiming any handle), so every surface shows a 10-char fingerprint (sha256("qalchat-fp-v1|handle|x25519|ed25519")[:10], base32, XXXX-XXXX-XX) meant to be compared out-of-band — voice, in person. A replaced keyserver key for an unpinned stranger still TOFU-pins silently: that residual risk is exactly what invites remove for people you actually invite.

Evidence (qalchat/test_invites.sh, 36 assertions, green, 2026-09-09): URI structure + QR PNG↔URI roundtrip (qrcode/zxing-cpp); accept pins the exact identity-file keys, 0600, mirrored source=invite; PNG-file accept; tampered bundle ⇒ exit 1, zero pin writes; messaging flows both ways over the pin; attack: relay restarted with substituted alice key ⇒ bob's send exits 4 with relay envelope count unchanged, genuine mail refused un-acked (exit 4), pin never replaced, full recovery after the keyserver is restored; unpinned stranger still works via TOFU; impostor invite verifies technically but its fingerprint differs from the real alice's.

v2 — on-chain DID anchoring of the pubkey (next). A QR invite still trusts the human handshake; the chain can make that continuous. v2 registers a did:qal-style identity object per handle on Qalnet — {handle, x25519_pub, ed25519_pub, rot_key, seq} in a Move-owned registry (mirroring qal_credit's object patterns) — so a client can resolve handle→key from chain state instead of the relay, verify the invite against the anchored key, and support signed key rotation (seq++ with the old key's signature) which converts "key change = alarm" into "key change = verifiable transition". Anchoring the fingerprint of each invite pin at accept-time would additionally timestamp the out-of-band trust event, giving conversations a notarized trust provenance chain: anchored DIDs for binding, invites for bootstrapping, hash-chain + Merkle anchors for message integrity — the keyserver becomes pure cache.

10. Encrypted file transfer (2026-09-09)

send-file <to> <path> moves a file (cap 8 MB) through the same ciphertext-blind relays, then the receiver auto-anchors the verified file hash on Qalnet via qalpipe — a transfer receipt with a real tx id.

alice: send-file bob report.pdf
  1. read file (≤8 MB), split into 256 KiB plaintext chunks
  2. seal EACH chunk: key = HKDF(convo_root, "qalchat/file/v1|fid|seq"),
     nonce random, AAD = convo_id|fid|seq  (per-chunk key; chunk cannot be
     replayed into another file or another position)
  3. upload CIPHERTEXT chunks → relay POST /files  (48h TTL; relay stores
     opaque blobs under data/files/<fid>/<seq>.bin — it never sees the
     filename, the plaintext, or any plaintext hash)
  4. seal a small FILE-MANIFEST message into the NORMAL conversation
     ratchet (ctr++, hash-chained, signed like every message):
       {file_id, name, size, sha256, chunk_total, chunks:[{id:"fid:seq",
        size, nonce_b64, ct_sha256, pt_sha256}, …]}

bob:   recv-files
  1. normal recv pulls the manifest (full gauntlet: msg_hash → sig → GCM →
     chain link)
  2. per chunk:  GET /files/<fid>/<seq>  →  sha256(ct) == ct_sha256 (catches
     at-rest tampering BEFORE any crypto)  →  AES-GCM open with chunk key +
     AAD  →  sha256(pt) == pt_sha256
  3. reassemble to a .part temp → verify WHOLE-FILE sha256 == manifest sha256
     and byte size — ONLY THEN rename into downloads/<me>/<fid>__<name>
  4. auto-anchor: qalpipe.anchor_file() → DataAnchor on qalnet-dev-1 → the
     `files` index records {sha256, tx, object_id}; chunks are swept off the
     relay (48h TTL is the backstop)

Failure at ANY layer ⇒ rejection (exit 3) and nothing is ever reassembled: the .part temp is removed, the download index is untouched, and recv-files re-scans history so a fixed/re-sent file can be retried later (manifests live in the conversation, not in a transient queue). files lists received files with anchor txs; --json for tooling.

Trust model: the relay is untrusted at every step — chunk integrity is checked against the manifest's sender-committed hashes (which travelled encrypted and signed through the conversation), so a tampered stored chunk is rejected with CIPHERTEXT HASH MISMATCH before decryption is even attempted (proven in test_files.sh §3). Chunk keys are domain-separated from message keys, so file transfer and chat never share key material. Honest limits: same ratchet root as messages (§2 limitation 1 applies), 8 MB is a Phase-0 policy cap (bump MAX_FILE_BYTES/MAX_FILE_CHUNKS/MAX_CHUNK_BYTES together), and the 48h TTL means the receiver must collect within two days.

11. Decentralised operation (2026-09-09)

qalchat has no servers, only relays — and relays are cattle, not sacred.

Multi-relay (--relays r1,r2,…). Overrides --relay. Send walks the list in order: primary first, next relay on failure (envelope lands on exactly one relay — no fan-out spam). Recv (and export) polls ALL of them and dedupes by envelope id — the same relay reachable under two URLs (127.0.0.1 / localhost) is processed exactly once, proven in test_files.sh §5. Key resolution walks the list too, and the TOFU/invite pin (pin_peer) hard-fails on ANY key change, so fetching a peer key from a different relay cannot silently substitute keys. Unreachable relays are skipped with a warning; all-unreachable is an error. send-file rides the same fallback for both chunk upload and manifest.

Self-host. A relay is one stdlib process with zero configuration:

python3 qalchat/qalchat.py serve-relay --host 0.0.0.0 --port 8830
# several INDEPENDENT relays on one machine:
python3 qalchat/qalchat.py serve-relay --port 8831 --state-dir qalchat/data/relay_state_2

No account, no domain, no fees: point clients at it with --relay / --relays. The relay is ciphertext-blind and stores no identities — losing it loses nothing but availability (24h window for un-collected mail).

Mesh ferry (export-env / import-env). The air-gapped path, shipped:

# online machine (or the ferry device itself) — pulls bob's unread mail:
qalchat.py export-env bob --as bob --out /usb/ferry.jsonl
#   → writes the SEALED envelopes to the file and ACKS them off the relay:
#     custody moves to the ferry file; the carrier cannot read any of it
# air-gapped machine (identity restored there via seed/recover):
qalchat.py import-env /usb/ferry.jsonl --as bob
#   → full verification gauntlet, zero network contact; history advances

Re-import is rejected (the chain-link check treats already-applied envelopes as gaps ⇒ exit 3), so envelopes apply exactly once no matter how many times the file is replayed. Together with seed-derived identities (§qal-derivation) this is complete offline operation TODAY: init from seed on device A, carry the words to device B, ferry mail on USB — the chain anchors integrity when you next touch a network, and never needs to before that.

What decentralisation means here, concretely: no identity provider (keys are local, invites are out-of-band), no name service (handles are portable, convo_id contains no relay URL), no payment intermediary (loopd is self-hostable, graduates to on-chain qal_credit), no data host you must trust (ciphertext-blind relays, any number, swappable per message), and no required connectivity at all (ferry). The federation sketch in §4 (envelope gossip between relays, handle homing) remains the next step — multi-relay clients already speak the shape it needs.