A working case study: curatorial.app, a multi-model, multi-agent AI platform for curators, designed, evaluated and operated end to end.
PORTFOLIO FOR
AI Strategy Consulting → AI Solutions Architecture
At 10–30% penetration, before a modelled 30–50% uplift from power users who bring their own key. Built bottom-up from public data, e.g. about 12,400 full-time US museum curators (BLS, 2025).
Largest reachable pool, lowest acquisition cost, fastest feedback loop. Reached through content and word of mouth.
Higher willingness to pay, but 6–12 month procurement. Enter bottom-up: individuals adopt first, the team follows.
Expands the addressable market once the product and the proof exist.
With roughly fifteen thousand reachable users, paid advertising reaches the wrong people. In a niche, the product reaches curators through curators.
The first win must cost nothing: upload two or three sources, run one analysis. The free tier runs on a cheap open-weights model with caps, plus a few taster queries on a better one.
One activation milestone (two uploads and one feature used), a target of under 48 hours to reach it, and a free-to-paid target set before launch.
| Free | Pro | Scholar | Team | Enterprise | |
|---|---|---|---|---|---|
| Price / month | $0 | $19 | $39 | $79 | custom |
| Editors included | 1 | 1 | 1 | 5 | unlimited |
| Read-only viewers | unlimited | unlimited | unlimited | unlimited | unlimited |
| Commenters | no | yes | yes | yes | yes |
| AI model class | open-weights | standard | standard + advanced | advanced | private |
| Signature unlock | taster queries on a better model | taster of writing modes | writing modes, style-guide upload, BYOK | shared research pools, shared style guides, roles | SSO, audit logs, dedicated hosting |
Public list prices. Model class follows the tier: cost per user rises only where the value does.
Give viewers away. Charge for editors.
Unlimited viewers on every plan, including Free, turns each shared project into an invitation.
By design, viewers read and comment; generating, uploading and indexing are editor actions. Viewers therefore cost almost nothing to host, and the expensive actions sit behind a paid seat.
A colleague invited to review a project has to create an account to take part. They see the product on real material, which is the cheapest acquisition there is. Needing a second editor is the natural step to Team.
Most viewers will never pay. That is accepted: they are the growth channel, and the cost ceiling is designed into the permission model, not left to hope.
No fine-tuned model needed: a growing corpus of curatorial material that any model can use through retrieval.
Curators do research-heavy, deadline-driven writing and planning. Collections systems have no AI; general chat tools have no grounding in the curator's own sources.
Market, segments, free tier, tiers, team model, moat.
RAG, multimodal, model tiers, BYOK.
Real-route tests, baselines, honest coverage.
Cost, monitoring, errors, bots, lifecycle, data.
Security, domain purity, decision records.
Slide 31 names what I would do differently. Slide 32 turns the case study into an engagement approach.
The writing flow is Socratic by design. It reads what the curator wrote, asks one question tied to a specific phrase, and only then helps structure and draft. The aim is to tease out and sharpen the writer's own ideas, not to generate text on their behalf.
Each question records its move type and what it was grounded in.
Documents, images, and audio recordings. Speech-to-text was tested across English, Japanese and Arabic.
Everything lands in the same retrieval layer, so questions and summaries cite recordings the same way they cite papers.
Scripted panel conversations rendered to speech. Voice choice came from a blind listening test of four candidates.
All of this is user-facing today. Models as of October 2026: MAI-Transcribe-2 (speech-to-text), MAI-Voice-2.1-Flash (text-to-speech).
Vectors live in the same Postgres as the data they describe. Search runs under the same row-level security, so one user's query cannot see another's library. (Moved from a separate vector database in March 2026.)
Indexing coordinator, retrieval service and embedding service: three small single-purpose modules instead of one pipeline class.
Retrieval is tested directly: 24 curatorial queries against a seeded corpus, scored on whether the right source lands in the top three.
Trade-off, accepted deliberately: changing the embedding model forces a full re-index and re-calibration of every similarity threshold, so it is treated as an architectural decision, not a config tweak.
Every call passes through one wrapper: usage, cost, user and feature are attributed automatically.
Model tier follows subscription tier; individual features can override with a better-suited model.
Swapping a model is a configuration change, tested before release.
Evals call the application's actual routes with real authentication and real tools, not a bare model, so tool selection and permissions are part of the test.
The grading model is pinned and never the model under comparison. The request-visible model override is gated by a shared secret.
A baseline is the median of repeated live runs; a change is flagged when the score drops by more than a tenth.
Every count test failed with 'permission denied'. Instead of patching permissions, I isolated the permission layer, proved it correct, and found a paid-tier feature gate the test user had never been given.
A notebook generator scored 0–10 out of 100 on groundedness. Retrieval was reading an unpopulated column. After the fix: 0.96. Likely affected every earlier output, not just the test.
One route returned a generic error on recoverable model output because it caught the SDK's exception type, not the wrapper's. Found by the harness, fixed, same class checked elsewhere.
A style-extraction route failed its schema on 6 of 6 live calls. It is excluded from baselines and labelled, rather than quietly recording failure as 'normal'.
Breadth is ahead of depth, and the slide says so.
A single function prices every call and feeds both billing and telemetry, so reconciliation is a "did we miss a call site?" check, never "do two calculations agree?".
Zero-token responses are flagged instead of silently billed as free. Unknown models fall back to a default rate and raise a visible warning.
Each call carries a canonical feature name, so cost per capability, not just per provider, is answerable. Model choice is a cost/quality decision with a test behind it.
One collector receives telemetry from every component and fans it out; errors go to a self-hosted, Sentry-compatible tracker.
Serverless handlers flush telemetry before returning, or traces are lost when the function freezes.
Dashboards and alert rules are kept as code; alert routing is the part still being finished (slide 31).
// what the client ever sees { "code": "QUOTA_EXCEEDED", "message": "Monthly limit reached" } // what stays server-side, in logs only stack trace · SQL · provider response · user data
Masking is a design rule, not a habit: it is checked by a lint rule and a pre-push gate.
Authorisation is enforced by the database itself. Application code is the second line of defence, not the first.
The bypass-everything database role is reserved for trusted internal paths with an explicit user identity, and is lint-guarded from client code.
Customer API keys are encrypted in a database vault and decrypted per request. They are never environment variables and never logged.
Clients receive an error code, never an internal message. Privileged database functions have explicit grants reviewed before use.
Bots authenticate as their own principals with signed tokens, not by impersonating a user.
Secrets are validated at boot: a bad configuration fails fast, not on the first live request.
The customer-service bot answers from a knowledge base, and an eval checks it does not invent features.
Lifecycle messaging is deliberately deterministic: a queue and a fire-and-exit runner, no model in the send path.
The same business data supports product-led growth: onboarding, behavioural segments, upgrade prompts.
Pipeline built and exercised with synthetic customers; no customer data appears in this deck.
One database means one security model, one backup story and one place to reason about data. It also means no staging copy today: a pre-launch trade-off named on slide 31.
// one place for role rules
RoleChangeValidator.canPromote(actor, target, team) {
actor.role === 'owner' && actor.status === 'active'
&& target.role !== 'owner' && sameTeam(target, team)
}Every line of code was written by LLMs, under my oversight, planning and guidance. I review, specify and decide.
That makes the guardrails the real engineering product: they are what turn fast generation into dependable software.
What: Inference, queue and bots moved from several vendors to one small server.
Alternative: Stay split across a managed app platform, a hosted queue and the bot host.
Trade-off: One pipeline and lower fixed cost, in exchange for a single point of failure.
What: A stalled worker can wake after recovery and overwrite it. Writes now check the claim timestamp.
Alternative: An id-only guard cannot tell the stale write from the legitimate one.
Trade-off: Slightly more code per state machine; recovery can no longer be silently undone.
What: Documented as a known limitation in May with options laid out; replaced by row-locked atomic deduction in August.
Alternative: Serialisable transactions, ordered queues.
Trade-off: Accepted a small risk first to avoid lock contention; paid it down once reviewed.
Inventory AI use cases, call sites, data flows and cost drivers.
Model choice as a decision: criteria, test, decision, review date. Build, buy or configure.
Small eval set per use case and baselines before rollout; human-in-the-loop designed in.
Cost attribution, error and abuse guardrails, review cadence.