The plan

01

Requirements

Two axioms, and everything else traces back to one of them, or to the unglamorous third axiom of anything self-hosted: the service has to survive contact with the internet.

• A1 — Anonymity. No identity (username, email, phone, IP) is ever linked to stored data. The account is a random key; there's nothing else.
• A2 — End-to-end encryption. The server has to be architecturally unable to read user data. Not just that it doesn't read it: it can't, even under a full database compromise or a malicious operator.

Everything else is derived requirements, grouped by what they defend:

Non-goals: account recovery, server-side merge or any plaintext processing, building an actual Mihon/Tachiyomi client (this page designs the client's behavior without building one), multi-blob sync, federation, sharing, and an admin UI.

02

Threat model

A threat model is a budget: where the design spends effort, and where it deliberately doesn't. The data being protected is manga reading history, which is low-stakes for most people. It's treated as sensitive anyway, on purpose, because the point of this project is learning to build the strong version properly.

One residual risk worth stating plainly: a malicious live operator can still observe request timing and sizes, serve stale data, delete data, or refuse service outright. End-to-end encryption hides content, not availability or metadata. Self-hosting doesn't change that; it just moves who you're trusting with it to yourself.

03

Cryptography

Everything derives from one client-held secret: the root key. The server never sees it in any form.

Key format

32-character Crockford base32: 31 random payload characters (155 bits of entropy), plus one Damm-style check character that catches every single-character typo and adjacent transposition locally, before it ever hits the network. It's displayed as 8 dash-separated groups of 4, and a QR code is the primary way to move it device to device.

The original idea was a 16-digit key (53 bits), and the math is why it got dropped. A fast hash cracks all 10¹⁶ keys against a leaked database in about 6 days on one consumer GPU. Even with a memory-hard KDF, a popular instance's leaked DB would let a GPU farm start popping random accounts within days, because the attack is inherently multi-target: the client has to find its own account from the key alone, so any salt has to be global and public. That's a structural property of "the key is the whole credential" designs, not an implementation gap. At 155 bits, the same attack needs longer than the universe has existed. Key stretching becomes unnecessary, and a single cheap HKDF is enough.

Key derivation

How the root key fans out via HKDF-SHA256 into the account ID, signing key, and encryption key:

root key  (client-held, never transmitted)
   │      canonical form: 31 payload chars, uppercase ASCII
   ▼
[extract]  K_master = HKDF-Extract-SHA256(salt = "shiori/v1/kdf-salt", ikm = payload)
   │
   ▼
[fan-out]  HKDF-SHA256-Expand(K_master, info, 32 bytes) — domain-separated:
   │
   ├── "shiori/v1/account-id" → account_id    (sent to server, DB primary key)
   ├── "shiori/v1/auth-seed"  → Ed25519 seed  (public key sent, private stays)
   └── "shiori/v1/data-key"   → K_data        (AEAD key — never leaves client)

Request authentication

Ed25519 signatures, not HMAC. An HMAC design would mean the server stores a shared secret, so a DB thief could write, overwrite, or delete data as any user. With Ed25519, the server only stores the public key: a DB thief can read the ciphertext they've already exfiltrated, but can't forge a signature.

Payload encryption

XChaCha20-Poly1305, key K_data, with a fresh random 24-byte nonce on every encryption. It's chosen over AES-256-GCM because several devices share one key with no channel to coordinate nonce counters. XChaCha's 192-bit nonce makes independent random nonces safe by construction, where GCM's 96-bit nonce would risk collisions at scale. Compression happens before encryption, and the client then pads the plaintext to the next 4 KiB boundary to blunt the size signal that compression creates.

04

Wire protocol

HTTP + JSON, all paths prefixed /v1. Every endpoint, including account creation, is signed with the account's Ed25519 key. There are no bearer tokens and no cookies.

The signature covers everything that gives a request meaning, so a captured signature can't be spliced onto a different request:

shiori-request/v1        ← fixed context line
{METHOD}                 ← uppercase, e.g. PUT
{PATH}                   ← path only, e.g. /v1/sync (no host, no query)
{TIMESTAMP}              ← exactly the header value
{NONCE}                  ← exactly the header value
{SHA256_BODY_HEX}        ← lowercase hex of SHA-256 of the raw body bytes

Verification runs in a fixed order:

1. Reject if the timestamp is more than 300s from server time, returning 401 timestamp_out_of_range. This check runs before any account lookup, so it leaks nothing.
2. Look up the account, verify the signature, check the nonce. All three failure modes (unknown account, bad signature, replayed nonce) return the exact same 401 unauthorized: same status, same body, and, best effort, the same timing.
3. Record (account_id, nonce) for 10 minutes as one atomic insert; a primary-key conflict is the replay verdict.

Account lifecycle, entirely server-side and enforced in the auth path itself (not just by the background sweep):

POST /v1/account
      │
      ▼
[provisional] ──── 5 min, no sync ────▶ deleted
      │
      │ confirmed (first successful GET|PUT /v1/sync)
      ▼
   [active] ◀── every auth'd request, bumps last_active_at
      │
      ├── 90d without any authenticated request ──▶ deleted
      │
      └── DELETE /v1/account (immediate) ──────────▶ deleted

The full endpoint-by-endpoint reference, request/response bodies and every status code, is in §09 below.

05

Architecture

One Cargo workspace, cut along what knows about what, not file size:

shiori/
├── Cargo.toml            (workspace)
└── crates/
    ├── shiori-core/      pure logic, no I/O
    ├── shiori-server/    the binary: HTTP + storage + sweeps
    └── shiori-cli/       reference client

shiori-core holds key derivation, blob format, and canonical-string signing, with no async, no HTTP, no database, no hidden clock or RNG. It's the same code a client needs, which forces the crate to stay honest. shiori-server is the axum binary; its security chokepoint is an auth extractor that runs the full verification order and hands handlers an already-authenticated account. Handlers have no other way to touch storage.

Storage is three tables, accounts, blobs, seen_nonces, via SeaORM against SQLite (the default: one file, WAL mode, trivial backup) or Postgres, picked by a connection string with no code changes. The absence of columns is itself a privacy feature: no email, no IP, no user-agent, no per-request log.

A single background sweep, on a 60s interval, removes expired provisional accounts, inactive accounts past 90 days, and stale nonces, each delete backed by an index on the columns it filters. But that sweep is lazy enforcement; the authoritative check lives in the auth path, so an expired account fails auth immediately, even before the sweep gets around to it.

06

Client

This section is design only; no client ships yet. But a sync API designed without thinking through the client is fiction, so every server-side choice above assumes this behavior.

A client holds the root key in secure storage, caches the keys derived from it, and never sends the root key, K_data, or the Ed25519 private key anywhere. Creating an account is immediately followed by a first push, which, when it succeeds, is what confirms the account inside its 5-minute provisional window.

Because the server never merges, pushing is a retry loop bounded by the version the client last saw:

loop:
    blob = encrypt(pad(serialize(snapshot)))
    PUT /v1/sync { parent_version: parent, blob }
    ├─ 200 {version}   → last_synced_version = version; done
    └─ 409 {current}   → theirs = decrypt(current.blob)
                          snapshot = merge(base, snapshot, theirs)
                          base = theirs; parent = current.version
                          retry (bounded; then surface an error)

The merge itself is three-way and field-wise, biased toward never silently losing reading progress: library entries union, chapter read-state only moves forward (mine.read OR theirs.read), progress markers take the max, and a deletion only wins if the other side left that entry unmodified.

A client also persists the highest version it has ever seen, and treats a lower version from the server as a rollback warning rather than data; that lets it detect a misbehaving server, though it can never prevent one. There's no device list and no revocation. Evicting a device means generating a new key, pushing the library under it, and deleting the old account.

07

Roadmap

Milestones are ordered so the pure core comes first, every milestone ends with something runnable, and hardening is its own step rather than an afterthought.

08

Deployment

Two supported ways to run it: a Dockerfile or a NixOS module. Both start from the same binary and the same env-first config, and differ only in packaging.

shiori-server speaks plain HTTP and always will; TLS termination is explicitly a reverse proxy's job (Caddy, nginx, anything that terminates TLS and forwards plain HTTP). This matters more than usual here: account_id is a stable pseudonym derived from the root key, so leaking it in cleartext lets a network observer link a device to an account over time, even though the payload itself is end-to-end encrypted.

On the SQLite backend, which both the Dockerfile and the NixOS module default to, there's exactly one file worth backing up: the database. Everything else is reproducible from source. A restored backup is indistinguishable from the server having been offline for a while, since the two TTLs keep working the same way against restored timestamps.

For the full walkthrough, config reference, and ready-to-run commands, see the self-hosting guide.

09

API reference

The complete machine-readable contract lives in openapi.yaml (OpenAPI 3.1); drop it into Redoc, Swagger UI, Postman, or a codegen tool. Where this page and that file disagree, the file wins. What follows is the human-readable version.

Endpoints

Errors

Every non-2xx response uses the same envelope:

{ "error": { "code": "unauthorized", "message": "human-readable, no secrets" } }

POST /v1/account

Registers account_id + Ed25519 public key. scheme pins the key-derivation version to this account forever; today only 1 is accepted.

Request body

{
  "scheme": 1,
  "account_id": "9f2c...32 hex...",
  "public_key": "base64url, 32 bytes"
}

201 response

// 201 — provisional
{ "expires_at": 1730000300 }

// 201 — already confirmed (idempotent retry)
{ "expires_at": null }

Idempotent at the protocol level for a retry with the same account and key on a live account, including concurrent retries, which resolve to the same 201 instead of erroring. A byte-identical resend, though, is a nonce replay, and gets the uniform 401. A row the auth path already considers dead (expired provisional, or 90-day inactive) is treated as nonexistent and gets recreated fresh instead of returning 409.

GET /v1/sync

Responses

// 200
{ "version": 7, "updated_at": 1730000000, "blob": "base64url..." }

// 204 — authenticated, but nothing has ever been pushed

PUT /v1/sync

Optimistic concurrency: the server only accepts a write whose author has seen the latest version.

Request body

{
  "parent_version": 7,
  "blob": "base64url of the full blob wire format"
}

Responses

// 200 — accepted
{ "version": 8, "updated_at": 1730000100 }

// 409 — someone else wrote first
{
  "error": { "code": "version_conflict", "message": "..." },
  "current": { "version": 9, "updated_at": 1730000050, "blob": "base64url..." }
}

On a 409, current is null only when the account was expired and re-created under the same key while this device still held old sync state. That's a rollback-shaped event, and the client has to surface it rather than silently merge.

Only an accepted push confirms a provisional account. A 409 or 413 leaves the 5-minute clock running, so a client that never lands a successful write loses the account at the provisional TTL.

DELETE /v1/account

Returns 204. Deleting an already-gone account yields the uniform 401, which a well-behaved client treats as success, at the cost of conflating "already gone" with "wrong key".