A developer's quiet notebook — practical notes from experiment to production
A vendor agent ships with evaluation built in; an agent you wired yourself does not. MAST builds a vocabulary for failure, AgentRx builds an auto-localizer, DRBench builds a testbed shaped like your data. All three are research-scale, so the builder borrows selectively. Part 3 of 'Locating Agent Failure'.
AI products are fundamentally different from traditional software. This post distills 12 counterintuitive lessons from 20 builders, the CC/CD framework for AI development lifecycles, and a 3-phase eval system - all sourced from Lenny Rachitsky's newsletter.
Self-Tuning Loop 4 steps (Generate → Capture → Analyze → Evolve) extracted as a universal module. Supabase DDL, diff capture utilities, analysis/evolution prompts, email/blog examples, GitHub reference implementation.
Evans 2026 deck notes - three chapters held together by one question: infrastructure boom rarely turns into value capture
A method-first read of the three papers that evaluate well-packaged agents (TRACE, DeepHalluBench, TRAIL): the altitude that rewrites the score, the one that verifies claims, the one that changes the substrate. Series Part 2.
Starting from the high-score illusion where the top-accuracy model ranks last on utility, this piece lays out why evaluation is moving to process and maps the methods of seven papers. Series Part 1.
Evans and Jensen both treat the model as a commodity, then split on whether value flows up to apps or down to infrastructure
I set the closest things beside mine to check if the spot was really empty. Where Khanmigo, Replika, and Brilliant part ways, and why the philosophy-chatbot line stays blurry. Chapter 4 of a serialized notebook.
After narrowing and paring, what was left fit no category. Too answerless for education, yet the same thing they did at Harvard. Was the blank spot an opportunity or a trap. Chapter 3 of a serialized notebook.
Listening notes on AI Frontier EP 98: the one-click era, why 'anything visible is open source,' and how the roles of VC and the university shift.
Giving shape to an AI tutor for morality, the first move was deciding what not to build. Why I narrowed to one trolley and shelved subscriptions, typing, K-12, and professional ethics. Chapter 2 of a serialized notebook.
AI governance is heading down a different path from typical regulation. EU, Korea, and US legislation, academic critique, market autonomous adoption, identity/KYC complement, compliance SaaS - five branches pointing the same direction, plus the 5 elements of governance taking shape.
The real buyers driving Deepfake Detection toward $15B aren't security teams - they're BFSI KYC departments. Four-SaaS breakdown, Hong Kong incident impact, and the Korean KYC 2.0 mapping through data.
Proof of personhood and AI agent identity are two layers of the same track. World ID, Passkey, DID adoption curves + Defakto, t54, Indicio funding + payment network entries - the entire identity track in one piece.
