For builders & security people · July 9, 2026

What we built, and how to break it.

A Personal Neutral Archive (PNA): a filesystem-as-database personal archive with append-only semantics, per-item provenance, consent classes, and, as of this week, a disposable SQLite projection layer with typed fact extraction. This page is the technical companion to the plain-words page. Our security posture starts with this sentence: the gaps are listed below.

1 · The invariants (everything else is implementation)

Files are canonical; every index is disposable. The archive is a plain directory tree: YAML manifests, per-item provenance.yaml, originals never edited. Any database must be regenerable from disk. The test is simple: delete every index, and the archive still reads with the basic file commands every computer ships with: ls, cat, grep.

Append-only. Nothing is deleted or overwritten: a new version lands beside the old one. Both the CLI layer and the direct-call layer refuse silent overwrite (tested).

One door (built — local; network bind next). All programmatic access passes a consent-enforcing MCP gate (pna-mcp, stdlib JSON-RPC over stdio) reading consents/policy.yaml, logging every call (allowed or refused) to an append-only access-log.jsonl. Sealed content (class C0) is absent from what callers see: a refusal would leak existence, so the gate never issues one. The gate makes sealed and nonexistent identical in text and response shape. Built and tested this session; not yet network-bound (that is Kurt's next seam).

Extraction produces derived files. Structure parsed from an artifact lands in the item's own derived/extraction-vN.yaml (generator named, source sha256 pinned, supersedes lineage recorded); the database only indexes those files. Rebuild re-reads YAML. It never re-runs OCR or a model.

2 · What exists and runs today

One Python CLI (stdlib-only, ~2,100 lines, runs on 3.9+; a vendored YAML engine keeps it working where PyYAML isn't installed; every YAML-touching test runs under both engines). One real archive (the founder's own, version 1, built July 8) plus the schema-v2 data layer (July 9). The suite has grown with every build since; current counts live on the build log, the site's one status inventory.

$ARCHIVE/
├── pna.yaml                    # owner, partitions, consent defaults
├── CATALOG.md · catalog.json   # regenerated; the tool owns these files
├── consents/
│   ├── policy.yaml             # identities + grants — the ONLY authorization truth
│   └── access-log.jsonl        # append-only; empty until the gate exists
├── _inbox/                     # the only write door for new material
├── _indexes/                   # 100% disposable; sync and backup skip it
│   └── catalog.sqlite          # the projection: items · assertions · FTS5 · views
└── <Partition>/<item>/         # 13 partitions (Receipts, Bloodwork, Senior-Care, …)
    ├── original/               # as it arrived; byte-identical forever (sha256-verified)
    ├── provenance.yaml         # source, acquired, consent class, derived/generator flags
    └── derived/                # regenerable; machine-made content labeled as such
CommandDoes
pna initScaffold an archive (any partition set; the CLI is domain-agnostic)
pna ingestDrain _inbox/ → partition, hash-verified copy (aborts on mismatch, source preserved), provenance written; refuses overwrites
pna extractNew: rules-first receipt parsing → derived/extraction-vN.yaml; a new extraction supersedes the old one, and nothing is overwritten; clean refusal on anything the grammar can't reliably parse
pna index --sqliteDrop + rebuild catalog.sqlite from manifests + extraction files (WAL, user_version=2, chmod 600)
pna verifyRe-hash every original vs manifests: drift / missing / foreign
pna verify --indexNew: recompute expected DB rows from the files, diff against the live DB (the index-poisoning detector); stale schema is a named state; it never reports false-clean
pna catalog · pna logRegenerate the human catalog · read the access log

The schema, verbatim

PRAGMA journal_mode = WAL;  PRAGMA user_version = 2;

CREATE TABLE items (
  partition TEXT NOT NULL, id TEXT NOT NULL,
  original_filename TEXT, sha256 TEXT, consent_class TEXT,
  acquired_date TEXT, ingested TEXT, status TEXT,
  title TEXT, derived INTEGER, generator TEXT,
  PRIMARY KEY (partition, id));

CREATE TABLE assertions (              -- one row per extracted fact
  item_partition TEXT NOT NULL, item_id TEXT NOT NULL,
  source_file TEXT NOT NULL,           -- lineage on every row
  kind TEXT NOT NULL,                  -- 'money.line' · 'money.total' · 'lab.value' · …
  observed_at TEXT, code TEXT, label TEXT,
  value_num REAL, value_text TEXT, unit TEXT, extra TEXT,
  FOREIGN KEY (item_partition, item_id) REFERENCES items(partition, id));
CREATE INDEX ax_kind_date ON assertions(kind, observed_at);
CREATE INDEX ax_item      ON assertions(item_partition, item_id);

CREATE VIRTUAL TABLE text_fts USING fts5(partition, item_id, source_file, content);
CREATE VIEW money AS SELECT * FROM assertions WHERE kind LIKE 'money.%';
CREATE VIEW labs  AS SELECT * FROM assertions WHERE kind = 'lab.value';

FTS feed rule: assertion labels + the text of derived .md/.txt files. Never originals, never the YAML. Builder and verifier share one row-mapping code path, so they cannot quietly disagree about what "correct" means. All SQL is parameterized. FTS5 is feature-detected with a real probe. Why SQLite at all: public domain, no listener, single file inheriting the box's whole security posture, format stable since 2004 with a written support commitment through 2050; the U.S. Library of Congress lists it as a recommended archival storage format.

3 · Enforced vs. recorded

Read this section the way we wrote it: as the attack surface. Version 1 is one real archive on one encrypted laptop. The consent gate is built and tested. It runs on the local machine, and it is not network-bound yet. These are the say-vs-do gaps that remain, stated before you find them:

ClaimStatus today
Originals byte-identical, append-only, hash-verifiedENFORCED — tested at two layers, re-verified every run
Index disposable / rebuildable / drift-detectableENFORCED — kill-the-indexes passed live; verify --index catches tampered, foreign, and stale-schema states (tested)
Extraction supersede + lineageENFORCED — v2 lands beside v1, index reads highest only (tested)
Consent classes C0–C5 per itemENFORCED AT THE GATE — every access through pna-mcp is class- and verb-checked and logged; C0 is invisible (sealed ≡ nonexistent, tested in text and shape). Raw filesystem access still bypasses the gate by design (that is the unplug test), so disk-level protection remains chmod 700 + full-disk encryption.
Per-partition consent overrides + verbs (read / write:<partition> / generate-brief)ENFORCED — partition- and verb-scoped grants tested, including a write-only aide who can add to one partition and cannot read a single item; expired / sunset / revoked grants never authorize.
Every access loggedENFORCED — the gate appends exactly one immutable line per call (allowed or refused); if the log cannot be written, the call is refused (no unlogged access). Tested, log-unwritable path included.
Replication + backups (3-2-1)PROVISIONING QUEUED — designed (Syncthing replica ignoring _indexes/, restic ×2, owner-held keys); until it lands, exactly one copy exists and our own report says so in bold
AI-generated content always labeledPARTIAL — ingest lacks a --generator flag; labeling AI items currently needs a hand-edit; nothing catches a forgotten one. On the attack list.

4 · Test it yourself

# scaffold a throwaway archive (keep real archives out of tests)
pna init --path /tmp/try-pna --owner you

# feed it a fake receipt, walk the whole path
printf 'olive oil 2L\t21.99\nbread\t4.50\ntotal\t26.49\n' > /tmp/try-pna/_inbox/receipt.txt
pna ingest  --path /tmp/try-pna --assign Receipts --yes --acquired-date 2026-07-01
pna extract --path /tmp/try-pna --partition Receipts --id 2026-07-01-receipt --category groceries
pna index   --path /tmp/try-pna --sqlite
pna verify  --path /tmp/try-pna --index

# ask it a question
sqlite3 /tmp/try-pna/_indexes/catalog.sqlite \
  "SELECT sum(value_num) FROM money WHERE kind='money.line'"

# the unplug test — the one that matters
rm -rf /tmp/try-pna/_indexes
grep -r "olive oil" /tmp/try-pna/Receipts/        # still answers
pna index --path /tmp/try-pna --sqlite            # rebuilds identically
pna verify --path /tmp/try-pna --index            # clean

The full suite covers both YAML engines, hash-mismatch aborts, overwrite refusals, drift detection in both tables, supersede lineage, stale-schema naming, and the receipt grammar's refusal paths. Acceptance tests 1–9, 12, 13 pass with recorded evidence; tests 10 (no listener) and 11 (gate scoping) now hold: the gate opens no socket at all (stdio), and grant scoping is enforced and tested. The gate adds its own conformance/behaviour tests plus an independent adversarial suite; the current counts across all of it live on the build log. Backup-exclusion (11's other half) still waits on provisioning.

5 · A bug story, because it shows you how we work

The v2 build shipped with 117 green tests, and all 117 missed a real defect. The receipt grammar's code comment promised a separator of "2+ spaces or a tab"; the shipped regex demanded two characters even when one was a tab, so an ordinary TSV-style receipt was refused. An independent verification pass caught it the same day by doing what test fixtures hadn't: feeding the tool input shaped by the world. The fix and two regression tests landed within the hour, and the build report records the miss as a miss, in its own amendment section.

That's the culture you'd be reporting bugs into. Say-vs-do gaps are treated as first-class defects here, including in our own claims: the table above is that principle applied to ourselves.

6 · What we'd love attacked

Our own adversarial pass has now run: an independent reviewer found four real gate defects (owner-identity spoofing, a write-commit crash, an owner emergency-lockout, a non-constant-time token compare); all four were fixed and regression-tested the same session, and the misses are recorded as misses in the build report. You're still welcome. The standing list, from our build reports:

TargetThe question
Filesystem exposure beside the gateThe gate enforces consent for access through it, but raw filesystem access bypasses it by design (the unplug test): reason about Syncthing/restic/local-model daemons that touch the tree directly, and about the still-local gate before Kurt network-binds it
Index poisoningverify --index is the defense; defeat it. Known soft spot: assertion identity is positional; a swap of two assertions' fields between positions is caught; the report describes it as two drifts and does not name it a reorder
Extractor, pathological inputThe parser is a regex grammar with no model behind it. Feed it hostile .txt (huge files, adversarial unicode/RTL labels, YAML-special characters) and prove it extracts correctly or refuses cleanly; it must never silently mis-extract
The forgotten hand-editAn AI-generated item that misses its manual provenance edit looks like a plain original, and nothing catches it: design the check we're missing
FK-declared-not-enforcedOur report reasons that orphaned assertion rows are still caught because verification is file-driven; the DB is never its source of truth. Test that reasoning
Dual-YAML parityRound-trip exotic YAML through both engines and diff: a silent divergence breaks environment-neutrality exactly where it's hardest to notice
Concurrent ingestItem-ID uniqueness check and write aren't atomic together; multi-device sync conflicts are a named open question
The future export pathConsent-scoped SQLite exports are designed. The build has not started. When they exist, prove a scoped export can't over-share, including FTS content leakage from out-of-scope rows

7 · Contribute, or report a problem

Everything enters through one door on the main site. We run no public inbox on purpose, and there is no tracker yet. Three doors:

Security findings → Write to us, kind “security.” Good-faith research is welcomed in writing: probe this system or our published code without destroying data, invading others' privacy, or disrupting service, report what you find, and we will not pursue legal action for the research itself, and we'll credit you unless you prefer otherwise. That's §15 of the site's Terms, in force now; a formal coordinated-disclosure policy ships with the public repository.

Building → Apply, kind “build.” The code is headed for a public repository with the launch; until then, specifications and build reports are shared with builders as they harden, and repo access follows a short trust ladder (a real conversation or a vouched referral; our screening policy applies the same consent-class idea to people). The jobs board on the main site lists real, fundable work.

Anything else → Write to us, kind “feedback.” A human reads every one.