From Zero to Running System

A runtime agent loop, built on top of the multi-provider work: a plan/execute/review pipeline with bounded self-correction, provider-agnostic by construction so the loop holds no model or transport detail of its own. A planner model writes a plan, an executor model runs the agent loop against the MCP tools, and a reviewer model checks the result and sends it back for correction when it falls short. Reliability was treated as first-class from the start: transient provider errors retry with backoff invisibly to the model, invalid tool calls are schema-checked and handed back to the model to fix, and a repeatability harness runs a task many times to measure success rate rather than trusting one green run. The first loop built on it keeps this website current from each project's changelog, readme and shipped version: the model decides what is stale and proposes a change-set, and code applies every edit deterministically, behind a safety gate that confines writes to the website directory and blocks git. The model plans; code writes.

The linear pipeline became one node type in a task-graph engine: DAG validation before any model spend, bounded concurrency via a hand-rolled promise pool, a per-run token budget as a circuit breaker, and fan-out/gather node factories with deterministic gather order. The first portfolio workflow fanned the code audit out over every enrolled project concurrently. On top of it, a run-control API on the existing gateway: list workflows, start a run, fetch a snapshot, and stream live node lifecycle events over SSE, all behind bearer auth because these endpoints trigger model spend.

Free-tier rate limits blocked the first portfolio run, so single models became ordered routes: preferred first, failovers behind. A transient provider error restarts the stage on the next model in the route; a real error is rethrown immediately. Telemetry records the attempted-model chain. In the same push, focused audit mode fixed the worst quality bug: a deterministic selector caps each audit at the highest-signal surfaces and a hard finalisation instruction makes ending a run without findings JSON a failure, not a silent clean pass.

A verifier node re-checks every audit result deterministically across four locked failure domains: output, completion, schema, and anchor. Every finding's anchor text must resolve verbatim against the file on disk or the finding is rejected; a failed verdict routes to one bounded re-run at reduced scope. The write path landed alongside it: apply is a replay of a saved, verifier-passed report, dry-run by default, gated to a filesystem allow-list, a clean working tree, and a mandatory build gate, with CRLF-aware matching after a line-ending mismatch blocked the first real candidate. Two real fixes went through the full path and were committed.

An opt-in challenger stage runs downstream of the verifier and can raise advisory cautions on a passing result, but never changes a verdict, widens apply eligibility, or triggers a write; the policy is written down, not implied. Workflow profiles then generalised the layer: a registry with a hardened contract puts a stable id and honest metadata over each workflow, validated before any model call. Two no-model detector profiles (documentation drift and changelog coverage) proved the graph carries more than audits.

A SQLite run ledger and queue (better-sqlite3, WAL) dual-written alongside the canonical JSONL telemetry, behind an async interface portable to a hosted database later. Store reads are opt-in with strict JSONL fallback; orphaned rows are retained and reported, never fabricated into history. Queue payloads are secret-free with credentials injected at drain time, the gateway's non-secret environment is pinned by contract, and an approval spine (grants bound to an exact apply fingerprint, issued by a fail-closed CLI) exists for any future unattended apply, which remains impossible by design.

A read-only, offline, no-model layer over the audit output: a workflow simulation report with structured evidence anchors, deterministic grouping, manual dispositions persisted in an append-only ledger, historical read-back, and a descriptive learning summary. Everything is a candidate, nothing auto-remediates. The website content audit also joined the graph as a first-class workflow: the proposal-first sibling of the code audit, fanning out over the enrolled projects with no apply path at all.

The dashboard grew native Workflows, Runs, and Graph tabs, then consolidated them into Studio: one hub organised around the run lifecycle, from the workflow catalogue through configuration, a live cost estimate priced from per-stage run history, launch behind an explicit spend confirmation, live observation with 4-second change-guarded refresh, and node drill-through to the findings report. Workflow and project catalogues moved into committed JSON registries so enrolment changes get git review. Next on the roadmap: breaking up the 12,000-line webview into modules.

The write path became an operator flow, each step designed first, put through an external adversarial review, then built: a dry-run diff surface in the code-audit panel, frozen approved-diff artefacts whose full hash is the authority key, atomic crash-safe reservation markers, then freeze, approve, and apply from Studio itself. Apply simulation closed the loop: the real apply engine and the target's own build gate run against the real tree, then always revert, so approval is only ever offered on a fix already proven to compile. The invariant is written down: nothing unapproved is kept. A fix-quality pass added a dedicated paid fix-model route and prompts that must stay within the evidence or decline with a reason, and on 4 July the first human-approved, artefact-bound write landed on a real project and was kept, build gate green. Around the milestone: finding-lifecycle badges, a sidebar status hero, a reusable model picker, fail-closed containment of project registry roots, and a one-script new-machine bootstrap. The dashboard shipped fast through this stretch: a new release most days for four days straight.

One fix, one gated write was proven. The next problem was a change too big for one edit. A decomposer now proposes a step plan while holding no write authority of its own: it can suggest, it cannot apply. Each step replays through the same gates and is committed individually with a provenance trailer, eligibility is independently re-checked at the apply seam rather than trusted from the plan, and a cumulative cap walks the real git history so a large change cannot be smuggled through as a run of small ones. The first live decomposition produced a validated three-step plan for about ten cents. Then, in a single day, the decision inbox: a read-only projection answering one question across every store, what actually needs me. Five slices shipped, then it stopped at five of six by explicit call, because the last slice needed a fleet that did not exist yet. Naming it deferred rather than leaving it in progress was itself the point: in-progress is what the cockpit reads, and a paused row sitting there misreports work in flight.

The verifier already assumed the model was unreliable. This phase assumed its input was hostile. Every audit pushes raw file bodies and website HTML into a prompt, so audited material is attacker-controlled the moment a dependency or a copied snippet lands in a repo. The injection fence gives every model call exactly one instruction channel: everything else travels inside a data block delimited by a run-scoped nonce, 72 random bits minted per run, and a block arriving with that nonce already in it refuses rather than degrades, because at 72 bits that is not coincidence. Checked rather than assumed: a before-and-after audit across three targets, about US$0.32, found no quality loss. Alongside it, two capability steps that both led with their limits. Notifications and a read-only mobile page reached the phone, pushing counts and class codes only, never content, on the rule that notification is signal, never authority. And a local Ollama fleet arrived in Studio, with execution-class recording shipped first so no local call could ever have gone unrecorded, and pipeline participation refused outright in v1 rather than allowed because it happened to be free.

Thirteen top-level tabs became seven: the old standalone Setup, MCP, Commands, Settings and Hooks became pills inside Config, and Development grew into a proper cockpit. The registry closed the injection track by proving its own coverage, enumerating every prompt surface with a source scan that fails if a dispatch site exists without a row, and it immediately made a real catch: a challenger stage shipped two days earlier without one. Then the phase earned its title. Testing whether granting apply authority would prompt, the command executed silently. The carve-out protecting the gate on every write to my code was enforced by none of its three supposed layers, and under bypass mode an ask rule is swallowed entirely. A permission gate that fails open is worse than none, because it is the one you have stopped checking. It was rebuilt fail-closed, returning an explicit allow, ask or deny per command. The uncomfortable part is not that it broke; it is that nothing would have told me.