// Topics / AI Operating Systems

AI Operating Systems

This hub collects the AI writing that is most useful for CTOs, founders, and engineering leaders who need to turn prototypes into reliable operating systems.

The archive is not about model hype. The through-line is operational: what to build, how to govern it, how to measure it, and where AI work fails when ownership is vague.

Start Here

Core Themes

Architecture

AI architecture is mostly about control surfaces. The model call is only one part of the system. The durable pieces are the routing layer, retrieval layer, validation path, observability, and rollback plan.

Useful next reads:

Governance

Good governance makes safe work faster. Bad governance turns every AI release into a committee meeting. The practical goal is explicit risk tiers, evaluation gates, and ownership for production behavior.

Useful next reads:

Economics

AI cost work is not just token optimization. The real metric is cost per useful outcome, including retries, evaluation, data work, human review, and incident response.

Useful next reads:

Teams

AI work breaks down when no one owns the boundary between platform, product, security, and operations. Strong teams make those interfaces explicit before scaling headcount.

Useful next reads:

Failure Modes

  • Treating AI as a feature instead of a runtime capability with ownership, telemetry, and rollback.
  • Measuring demo quality while ignoring cost per outcome and production drift.
  • Centralizing every AI decision until the platform team becomes a queue.
  • Shipping model behavior without evaluation cases tied to real workflows.

References

157 entries tagged “AI Operating Systems”

    Compliance Fails at the Vendor Handoff · 4 min You answer for an AI system you can only partly witness. Build a compliance interface to each vendor, with owners, deadlines, and drills. compliance governance ai Your AI Tool Needs an Adoption SLO · 4 min Seat counts can't justify an AI tool renewal. Define one adoption SLI, governed completions over eligible cases, with guardrails and a breach rule. ai operations metrics The Review Queue Is Your Real Agent Limit · 4 min Plan agent rollouts like capacity: risk-weighted review demand against effective reviewer-hours. Past the constraint, seats buy inventory, not throughput. ai operations teams Content Marking Is a Pipeline, Not a Policy · 4 min The EU AI Act marking deadline for existing generative systems is 2 December. Provenance survives only what your pipeline keeps, so test it in CI. compliance ai architecture Your Coding Agent Inherits the Engineer · 4 min A coding agent inherits every credential the engineer holds and none of the judgment behind them. Close that gap with a runtime boundary, not a ban. security ai operations Agent Scale Has a Company-Size Gap · 4 min Large firms report scaling AI agents; smaller ones sit flat. A small company needs one workflow with an owner, gate, boundary, spend cap, and off switch. ai strategy operations A Backup Model You Never Exercise Is Not a Backup · 4 min Vendor count is inventory; measured failover time is resilience. Shadow replay, a bounded canary, and a recovery clock with defined start and stop. ai reliability strategy The Handoff Tax · 4 min 80% report personal AI gains while EBIT impact stays flat. The handoff tax: task time down, throughput flat, queues rising at one nameable transition. leadership operations ai The Statistic Nobody Can Reconstruct · 4 min I tried to reconstruct '95% of AI pilots fail.' It isn't in the report it cites. Card the numbers that steer your decisions: licensed, expired, or retired. metrics ai executive Don't Book the Cut Before the Work Disappears · 4 min Expected AI headcount cuts outran reported ones last year. Let forecasts drive scenarios; book a saving only once the work has observably gone. leadership teams ai Autonomy Without a Demotion Path Is Permission Creep · 4 min Teams can describe how agents earn autonomy. Few can take it back. Write the transition contract: trigger, authority, mechanics, re-entry. ai governance reliability The Software You Didn't Buy Still Needs an Owner · 4 min Teams skip software purchases and build with agents. Each build is rational; the fleet is the risk. Agents compress upkeep but can't own it. strategy ai cost MCP Grew Up. Your Integration Debt Has a Clock. · 4 min The July MCP revision started a 12-month deprecation clock. Use it to find every agent-to-tool boundary, give each an owner, and test it. ai architecture platform-engineering The Rung Contract · 4 min Competency matrices describe skills. Apprenticeship needs a contract per rung: permission, evidence, reviewer, a promotion rule two managers apply alike. leadership teams ai Your Real Token Price Is a Cache Hit Rate · 4 min For input-heavy agent loops, effective token cost rides on cached-token share. Track your hit rate, and measure the cold-cache premium before you switch. cost ai operations The EU Gave You a Reprieve, Not a Pardon · 5 min The Digital Omnibus deferred the AI Act's high-risk duties, not its live ones. Build a deferral ledger: one row per system and obligation. governance compliance ai The Eval Inversion · 5 min Two frontier labs lost containment this summer, in eval environments. The more adversarial the workload, the weaker the walls around it. Invert that. ai security reliability The Junior Developer Cliff Is a Leadership Problem · 4 min AI eats the work juniors learned on. Rebuild apprenticeship around verification and ownership, or starve your senior bench. leadership ai teams Garbage Context, Confident Answer · 4 min Most AI failures are context failures wearing a model's face. Score retrieval with groundedness and margin, check freshness against live state. ai reliability architecture Geopolitical Model Risk Is an Engineering Constraint · 4 min Export controls, gated releases, and open-to-closed reversals make model availability an architecture-review line item. ai architecture strategy The Diffusion Gap: Why Pilots Don't Become Capability · 4 min Most AI pilots pass the trial and then die. Closing the gap takes ownership, cadence, and incentives. Write exit criteria as operating changes. ai operations leadership AI Insurance Will Ask for Evidence, Not Intent · 4 min As insurers exclude AI, protection tracks evidence, not intent. Your operating cadence is your audit trail. governance ai executive The Board's AI Oversight Problem Is Operational · 4 min Board AI oversight is operational: a named owner per system, a rehearsed halt path, an incident threshold, and a failure rate you can defend. governance ai executive Unsupervised · 7 min Producing the work can be handed to a machine. Answering for it cannot. What an organization values when machines do most of the making, what people are for, and four things to try this week. opinion ai strategy Shadow AI Is an Operating Problem, Not a Ban · 4 min Banning AI tools removes your visibility, not the tools. Make the governed path the fast path and pull usage into the control plane. governance ai security Power Belongs on the AI Roadmap · 3 min Grid power and GPU quota arrive on lead times of months to years. Put region, capacity type, owner, and fallback next to every committed AI ramp. ai architecture strategy Regulatory Divergence Is a Routing Problem · 4 min One global AI policy is wrong in every market. Tag requests by jurisdiction, data class, and user type, then route policy like cost and capability. governance ai privacy Reliability Is the Autonomy Ceiling · 4 min How much autonomy to give an AI agent: bound its failure rate with the rule of three, shadow mode, and fault injection, then price it by failure cost. ai reliability operations Your Vendor's Balance Sheet Is Your Risk · 4 min AI vendor risk you can price: compare the API price to a self-hosted floor, lock deprecation notice and export rights at signing, fund a warm fallback. strategy ai cost Agent Identity Is the New Control Plane · 4 min AI agents need workload identity, not shared API keys: SPIFFE SVIDs, RFC 8693 token exchange, and Vault leases make access scoped, attributable, revocable. ai reliability security Token Prices Fell. AI Bills Did Not. · 3 min Per-token LLM prices keep falling while AI bills climb. Measure cost per completed task, reviewer time included, instead of price per token. cost ai executive The Benchmark You Didn't Build · 4 min Use public LLM benchmarks as a shortlist filter, then decide on an owned eval: programmatic assertions, a versioned LLM judge, and paired per-case diffs. ai reliability metrics Leading Senior Engineers in the AI Era: Autonomy, Standards, and Accountability · 4 min Leading senior engineers on AI features: a written definition of done with an owned eval, a named failure mode, a rollback drill, and a paging threshold. leadership ai teams Sovereignty-by-Design for AI: How to Win Regulated Enterprise Deals · 4 min AI data sovereignty an auditor can test: BYOK, in-region decryption, and which copies key revocation kills versus which need signed delete receipts. ai privacy governance Agentic Systems at Scale: The New Reliability Contract · 4 min An AI agent reliability contract is real only where the control plane enforces it: scoped credentials, a deny-by-default tool gateway, a sandbox. ai reliability operations The Anti-Fragile AI Organization · 4 min A warm LLM fallback is a standing cost that needs an owner. Price portability per feature, and turn each vendor shock into a capability you keep. teams ai reliability The AI Strategy Stack: What Boards Mistake for Moats · 4 min Models and prompt scaffolding are not AI moats. The defensible layer is a correction loop on your own work, tracked by escalation rate, and it depreciates. strategy ai executive From Model Demos to Profit Engines: The CTO Playbook for AI Unit Economics · 3 min AI unit economics: route requests by value and risk, price in failure and rework, and measure margin per workflow instead of per model call. ai cost strategy The New Talent Stack: Product, Platform, and Applied AI Must Work as One System · 2 min AI organizations create leverage when product, platform, and applied AI are designed as one operating system instead of three kingdoms. teams ai platform-engineering The Executive Case for Local-First AI Infrastructure · 2 min When to run AI workloads locally and when to stay on cloud APIs: a placement rubric based on call frequency, data sensitivity, and fallback readiness. ai architecture cost The Post-Prototype AI Org: Operating Models That Survive Year Two Canon post — · 2 min Year-two AI failure usually comes from org-design mismatch, not model-quality mismatch. The handoffs are where the system slows down. ai teams leadership The Operating Cadence: Turning AI Leadership Interfaces Into Predictable Output Canon post — · 3 min Leadership interfaces decay without rhythm. A weekly metrics review, monthly outcome review, and quarterly architecture audit keep AI ownership real. leadership ai operations Designing the AI Leadership Bench: Roles, Interfaces, and Failure Boundaries Canon post — · 2 min Scaling AI needs a leadership bench: named owners for product, platform, applied AI, and governance, with failure handoffs rehearsed before incidents. leadership teams ai Decision Latency as a P&L Variable: The Leadership Metric Nobody Owns Canon post — · 2 min Slow decisions are a hidden cost. How to measure decision latency, from issue to decision to action, and cut it with clear ownership of decision classes. leadership metrics strategy The AI Vendor Negotiation Playbook for CTOs · 3 min A pre-meeting checklist for AI vendor contract negotiations: know your workload, switching cost, eval evidence, and exit plan before the vendor does. ai strategy cost How to Run an AI Incident Review That Changes Architecture, Not Slides · 2 min An AI incident review is done when it changes architecture, evals, alerting, or ownership. An eight-part template that ends in fixes, owners, and dates. reliability ai governance AI Roadmaps That Survive Contact With Reality Canon post — · 3 min AI roadmaps fail when they are sequenced around ambition instead of dependency, verification, and rollback cost. strategy ai leadership Hiring for AI Teams: The Operator Profile That Scales · 3 min The AI hires that scale are operators who handle ambiguity, systems tradeoffs, and verification pressure. Four traits and four interview questions. hiring ai leadership Technical Leadership in the AI Era: Throughput Over Trends · 3 min Technical leadership in 2026: anchor decisions in throughput, verification, and operability instead of chasing the latest agent framework. leadership ai teams Stop Building Internal AI Tools No One Uses · 4 min Internal AI tools die quietly when teams optimize for launch over workflow fit, output trust, and post-launch ownership. One tool failed, one stuck. productivity ai leadership Why Most AI Platform Teams Become the New Bottleneck Canon post — · 3 min AI platform teams become bottlenecks when they centralize decisions instead of capabilities. The warning signs, the metrics that expose it, and the fix. platform-engineering ai teams Build the System the Model Cannot Break Canon post — · 12 min A manifesto for AI-native organizations: twelve tenets across strategy, architecture, economics, and people, and the one test that matters in year two. opinion ai strategy The CTO Communication Protocol for AI Programs Canon post — · 3 min AI programs fail when engineers, executives, and investors hear different definitions of success. A communication cadence that keeps one true story. leadership ai executive AI Governance Without Bureaucracy · 2 min AI governance that works: tighter defaults, named owners, fast escalation, and a small control stack engineers can follow at 2 a.m. No new committees. governance ai security Measuring AI Progress Without Theater: A Board Scorecard · 2 min Most AI progress reporting confuses activity with value. Executive measurement should collapse around adoption, reliability, margin, and delivery speed. metrics ai executive AI Build vs. Buy in 2026: An Operational Cost Question · 3 min AI build vs. buy in 2026 turns on operational cost: who owns telemetry, fallbacks, and data residency. When to buy, when to build, and the hybrid default. strategy ai architecture AI Strategy Metrics: Margin, Risk, and Speed · 2 min Replace AI novelty metrics with three executive numbers: margin, risk, and speed. Give each a baseline, target, owner, cadence, and rollback path. ai metrics strategy AI Evaluation and Production Governance: A Maturity Model · 4 min A five-level maturity model for AI evaluation and production governance, from vibes-based deploys to CI eval gates, production sampling, and rollback. governance ai reliability Why Most Enterprise AI Architecture Fails in Year One · 3 min Enterprise AI architecture fails in year one when teams expect deterministic behavior from a statistical engine. Build failure boundaries and telemetry. architecture ai reliability AI Capital Allocation: What to Stop Funding First · 4 min Strong AI strategy starts with a kill list. If a project cannot defend margin, risk, or speed, it should not survive the next budget meeting. ai strategy cost AI Strategy for CTOs: It's Data Infrastructure · 2 min AI strategy for CTOs: the moat is the context pipeline, the eval loop, and the fallback path. Picking a model is the easiest problem in the room. strategy ai executive Why AI Agents Fail on Legacy Network Infrastructure · 7 min Many AI agent failures trace to the network: stale DNS, flat trust, retry storms, shallow queues. Zero-trust identity and per-step reliability fix them. agents infrastructure security Local-First AI Agents: Why Cloud-Heavy Architecture Fails · 7 min Agents that chain dozens of cloud hops pay in latency variance, cost, and failure surface. How to move routine inference local without a rewrite. agents infrastructure cost AI Startups in 2026: Where the Leverage Is · 3 min What buyers of AI startups reward in 2026: workflow ownership, data loops, integration depth, and predictable cost per outcome. Novelty stalls at pilot. startups ai business AI Security in 2026: Prompt Injection, Agents, and Defenses · 6 min Multi-step prompt injection and tool-based exfiltration target AI agents in 2026. The layered defenses, monitoring, and security review I use. security ai production AI Team Structures 2026: Central, Embedded, and Hybrid Models · 8 min Central, embedded, or hybrid AI team? How to decide where AI ownership lives, with roles, tradeoffs, platform-to-product ratios, and scaling rules. teams ai architecture AI Inference Cost Trends 2026: LLM Prices, September Update · 13 min LLM API prices now run from $0.10 to $10 per million input tokens. A sourced September 2026 price table, the trend since GPT-4, and where savings come from. cost ai performance AI Regulation Is Here: Treat Compliance as Engineering · 7 min AI regulation already shows up in procurement and security reviews. How to build compliance evidence into the delivery pipeline instead of bolting it on. compliance ai governance AI-Native Architecture Patterns 2026: Production Guide · 6 min AI-native architecture patterns for production: an AI gateway, a retrieval layer, an evaluation pipeline, fallbacks, cost control, and clear ownership. architecture ai engineering Building Reliable AI Agents in Go · 5 min How I build reliable AI agents in Go: bounded tools, schema validation at the boundary, idempotent state, and a supervisor loop with hard limits. agents reliability ai Video AI in Practice: Pipelines, Evaluation, Human Review · 4 min Video AI is practical for scoped workflows. A reference pipeline, the evaluation metrics that matter for video, and where human review still belongs. llm ai production AI Predictions for 2026: Bounded Agents, Routing, Governance · 4 min What I expect from AI in 2026: bounded agents in production, routing across models, MCP as the tool layer, and governance on the critical path. trends ai 2025 Year in Review: AI Stopped Being Special · 5 min The operational shift in AI during 2025: evals, governance in CI, cost routing. The novelty phase is over and the infrastructure phase has begun. year-in-review ai reflection AI in 2025: What Held Up in Production and What Didn't · 4 min The most important thing that happened to AI in 2025 wasn't a model release. It was the shift from 'what can it do' to 'how do we run it.' That's progress. reflection ai Scaling AI in the Enterprise Is a Management Problem · 4 min AI pilots work. What fails is going from five demos to fifty production features without an operating model. That's a management problem. business ai architecture AI Incident Response: Failures That Don't Look Like Outages · 4 min AI systems can return 200 OK while confidently wrong. How to detect, contain, and learn from AI incidents using proven incident response principles. incident-management ai reliability AI Technical Debt: Prompts, Evals, and Pipelines Nobody Owns · 4 min AI debt hides in prompts nobody owns, evals nobody runs, and data pipelines nobody watches. By the time you notice, every change feels dangerous. technical-debt ai engineering AI and Team Productivity: Shared Infrastructure Wins · 3 min Individual AI speedups are a distraction. The real gains come from treating AI as team infrastructure, embedded in docs, decisions, and onboarding. productivity ai teams Measuring AI ROI Without Lying to Yourself · 4 min Most AI ROI calculations are fantasy. Measure one workflow, count full costs, tie benefits to tracked outcomes, and report a range, not one number. metrics ai business AI Data Privacy Is a Plumbing Problem · 5 min AI privacy fails in the details: what gets logged, who can replay prompts, how long artifacts linger. Treat it as infrastructure, not a checkbox. privacy ai data AI Pair Programming: It's a Junior Dev, Not a Wizard · 4 min Treat AI coding assistants like a fast, literal junior dev: tight constraints, critical review, and no expectations of architectural insight. ai engineering productivity AI Workflow Automation: Let the Model Decide, Let Code Act · 4 min The trick to AI workflow automation is simple: let the model decide, let deterministic code act, and never confuse the two. devops ai agents AI Documentation Search: Versioned Retrieval and Citations · 4 min AI docs systems often retrieve the wrong version and never admit uncertainty. Build one with hybrid search, version filters, citations, and freshness jobs. engineering ai llm AI Product Metrics: Measure Task Success, Not Usage · 3 min Engagement metrics tell you people clicked, not whether your AI feature helped. Measure task success, correctness, and trust signals instead. metrics ai strategy When to Fine-Tune an LLM: Prompt and Retrieve First · 4 min Fine-tuning is the go-to move for teams who skipped the basics. Most of the time, better prompts and proper retrieval solve the actual problem. llm ai opinion AI Customer Support: Design for Handoff, Not Deflection · 4 min Most AI support systems are built to deflect tickets. The ones that work are built around escalation, grounding, and the idea that customers aren't idiots. ai production AI Data Pipelines for RAG: ETL With a Retrieval Layer · 5 min AI data pipelines are ETL with a retrieval layer bolted on. The discipline is the same as always: detect change, chunk intelligently, keep indexes fresh. data ai infrastructure Multi-Agent Orchestration: Four Patterns and Their Tradeoffs · 5 min Multi-agent systems are distributed systems with the usual coordination headaches. The four patterns I've seen work, and when each one falls apart. agents ai architecture AI Security: Prompt Injection and the New Attack Surface · 5 min AI systems are exposed APIs with real blast radius. The threats are injection, leakage, and tool misuse. The defenses are the ones we've always needed. security ai production Testing AI in Production: Shadow Mode, Canaries, Holdouts · 6 min Offline evals aren't enough. How I test AI features in production with shadow mode, canaries, holdouts, and automatic fallback, with Go code. testing ai production AI Observability: Green Dashboards, Wrong Answers · 4 min Uptime monitoring won't show an AI service returning confident garbage. The five signals I track instead: traces, quality, cost per outcome, safety, ops. observability ai production Model Context Protocol in Go: Building an MCP Tool Server · 7 min I built a Model Context Protocol server in Go with mcp-go. The protocol layer is clean. Auth, permissions, and write safety are still on you. agents ai go AI Governance That Works: Risk Tiers and System Cards · 3 min AI governance that blocks delivery is broken. How to make 'yes' safe and fast instead: risk tiers, one-page system cards, and living evidence. ai governance compliance Video Understanding AI: Transcript First, Frames Second · 4 min I built a video understanding pipeline for support call review. What worked: transcript first, scene-based frame sampling, timestamps on every claim. llm ai data AI Code Review Is Mostly Noise · 4 min Months of AI code review on real PRs: about 22% of comments get accepted. How I scope prompts, track hit rate, and keep it out of merge gates. engineering ai development Reasoning Models in Production: A Practical Guide · 7 min Reasoning models are slow and expensive. How I run them in Go services: complexity routing, async jobs, per-request budgets, and result caching. llm production ai AI Trends for 2025: Why Engineering Discipline Wins · 4 min The AI hype cycle is over. 2025 belongs to teams that make AI work in production: repeatably, measurably, and without burning money. ai trends strategy AI in 2025 Will Reward the Boring Teams · 3 min The AI advantage in 2025 goes to teams that ship measurable workflows. Where to focus: evaluation, cost attribution, lightweight governance. ai strategy 2024: The Year AI Got Boring (In a Good Way) · 4 min My 2024 in AI: the demo phase ended and production work began. What held up, what surprised me, and what stayed hard across a year of hands-on AI work. year-in-review ai reflection AI Infrastructure at Scale Is Just Infrastructure · 4 min AI infrastructure at scale is just infrastructure. Gateways, caching, workload separation, budgets, and circuit breakers solve the same old problems. ai infrastructure architecture Building AI Teams: Ownership and Evaluation Beat Hiring · 4 min Most AI team failures come from unclear ownership and weak evaluation, not missing talent. Team structures that work, who to hire, and the operating loop. ai teams hiring Picking an AI Model for Production (Late 2024) · 5 min No LLM is best for everything. Compare task fit, tail latency, cost per success, and schema compliance in a bake-off on your workload, then route. ai llm reflection AI Safety in Production Is Defense in Depth · 5 min Production AI safety works like cyber defense: assume breach, layer input, output, and system controls, and watch every boundary with real monitoring. ai governance production AI Agent Patterns in Go: Planning, Memory, Recovery · 7 min Single-prompt agents break on real tasks. Plan-execute-replan, orchestrated specialists, structured memory, and explicit recovery, with Go code. agents ai go LLM Cost Benchmarking: Measure Cost per Completed Task · 4 min Price-per-token is the least useful number on your AI bill. Real cost benchmarking starts with your workload, not a provider's pricing page. ai cost llm AI for Technical Documentation: Drafts Yes, Ownership No · 3 min AI is a decent drafting assistant for technical docs and a poor substitute for ownership. Draft with AI, review in the same PR, and ground doc search. engineering ai developer-experience AI-Assisted Code Migration: Lessons From 200K Lines of Go · 4 min I used LLMs to help migrate a 200K-line Go codebase. The mechanical parts went fast. Everything else was still hard. ai technical-debt go How I Test LLM Features: Three Layers, One Cadence · 5 min LLM outputs are non-deterministic. That doesn't mean you can't test them rigorously. Here's the layered testing approach I use in production. llm testing ai Small LLMs in Production: Use the Smallest Model That Works · 3 min Most production LLM tasks are classification and extraction that don't need GPT-4. Where small models win, where they fail, and how to route. llm ai performance Context Window Budgets: Stop Stuffing Your LLM Prompts · 4 min Bigger context windows don't excuse sloppy prompts. Budget tokens per section, pin anchors, retrieve less, and measure quality against context size. llm ai architecture LLM Function Calling Patterns That Survive Production · 7 min Function calling is how LLMs touch real systems. Treat tools like APIs, arguments like untrusted input, and the model like an intern with root access. llm ai go Claude 3.5 Sonnet Analysis: Cost, Coding, and Model Routing · 5 min Four days with Claude 3.5 Sonnet: coding quality, where Opus still wins, what Artifacts adds, and why it changes model routing for production. llm ai reflection Enterprise AI Compliance Without the Theater · 5 min Enterprise AI compliance as architecture: a data inventory, model registry, audit logging, and risk tiers that let low-risk use cases ship in days. ai compliance business Why Enterprise AI Pilots Stall Between Demo and Production · 4 min Most enterprise AI pilots stall between demo and production. The blockers are organizational: ownership, security review, data access, governance. business ai strategy Building Voice AI: Latency, Interruptions, and Narrow Scope · 5 min Voice AI is ready to ship. The hard parts are latency, interruptions, and knowing when voice is the wrong interface. Here's how I approach it. llm ai go GPT-4o Changed the Interface, Not the Hard Part · 4 min GPT-4o puts text, vision, and audio in one model. That removes pipeline glue, but transport, devices, and consent stay hard. How I'd evaluate it. llm architecture ai Most AI Developer Tools Are Not Worth Adopting Yet · 3 min AI developer tools are multiplying, and most add complexity without removing friction. The filter I use to decide what earns a spot in the stack. ai developer-experience opinion Agentic Workflows in Production: Constrain the Blast Radius · 5 min AI agents that take actions carry real blast radius. Policy allowlists, structured workflows, idempotent steps, tracing, and a shadow-mode rollout. agents ai production Multi-Model LLM Routing and Fallbacks in Production · 4 min Betting on one LLM provider is a database with no failover. How I route between models, chain fallbacks, and log which model served each request. ai architecture llm Claude 3 First Impressions: Picking Opus, Sonnet, or Haiku · 4 min Claude 3 shipped as three models. First notes on Opus, Sonnet, and Haiku quality, vision, and context, and how I route work between the tiers. llm ai LLM Evaluation: Stop Shipping on Vibes · 5 min LLM evaluation that catches regressions before users do: a realistic test set, layered checks, offline and online evals, and deploy gates. ai llm testing Architecting AI-Native Applications (Without the Delusion) · 7 min AI-native apps differ from a model bolted onto a CRUD app. The layers, confidence routing, fallbacks, and feedback loops I use, with Go code. architecture ai engineering AI Engineering Is Its Own Discipline Now · 4 min AI engineering is not ML research with a product hat. It is the work of making models behave in production, and it needs its own hiring profile. ai hiring llm 2023 Year in Review: LLM Integration, Payments, and Go · 4 min Looking back at 2023: LLM integration work, Go backends for payments, production AI deployments, watching the AI platform shift, and what stayed hard. year-in-review ai reflection Scaling AI Features: GPUs, Rate Limits, and Backpressure · 4 min GPUs are scarce, rate limits are hard ceilings, and AI demos collapse under real traffic. The distributed systems patterns that keep AI features up. ai infrastructure architecture GPT-4V Use Cases: Five That Work, Three That Don't · 5 min Weeks of testing GPT-4V on real workflows: receipts, UI review, alt text, and diagrams work; counting, measurements, and tiny text don't. Plus cost tips. ai llm business OpenAI Assistants API: Two Weeks of Real Use · 4 min Two weeks building internal tools on OpenAI's Assistants API: quick wins with retrieval and code interpreter, opaque internals, and runs that hang. llm ai go OpenAI DevDay 2023: What It Means for Build vs Buy · 4 min GPT-4 Turbo, the Assistants API, and GPTs moved OpenAI further up the stack. What that means for teams deciding which AI plumbing to build or buy. llm ai architecture AI Coding Assistant Productivity: Three Months of Numbers · 5 min Three months tracking Copilot and GPT-4 on real Go work: 25-30% faster on boilerplate, no gain on debugging, and a 15% review tax on AI-written code. ai developer-experience productivity LLM Security: A Field Guide for People Who Ship Things · 6 min LLMs bring security failure modes most teams aren't defending against. Prompt injection, data leakage, tool abuse, and cost attacks are exploitable today. security llm ai Responsible AI Is Operational Risk Management · 3 min Responsible AI fails as a committee and works as risk management: know the blast radius, test the failures you fear, disclose AI use, name an owner. ai security governance AI Technical Debt: Prompts, Models, and Evals Nobody Tracks · 4 min AI features pile up debt in unversioned prompts, unpinned models, unvalidated data, and missing evals. Warning signs and how to pay it down. ai technical-debt engineering AI Agent Architecture Patterns for Production · 6 min Agent demos impress. Production agents mostly don't. Planning, memory, least-privilege tool access, and evals: the systems design that decides what ships. ai agents llm AI Product Strategy: Start With the Problem, Not the Model · 4 min Every 2023 roadmap has an AI feature, and most start from the model. A filter and five questions for picking AI features users will actually value. ai strategy startups LLM Observability: Monitoring Quality, Not Just Uptime · 5 min Traditional monitoring says the service is up. It won't tell you the model started returning garbage last Tuesday. How to actually observe LLM systems. observability llm ai What I Learned Building AI Features Into a Fintech Product · 5 min Shipping AI transaction categorization in a fintech product: two-day demo, six weeks to production. Eval sets, input normalization, fallbacks, rollout. ai strategy business LLM Cost Optimization: The Basics Most Teams Skip · 4 min A 12-person startup got a $14,000 OpenAI bill. The fix wasn't a FinOps tool: routing, caching, max_tokens, prompt trimming, per-feature cost tracking. ai cost llm Embedding Models Compared: Quality, Cost, and Latency · 6 min ada-002, instructor-large, and all-MiniLM-L6-v2 on a 150-query retrieval eval: precision, MRR, latency, index size, cost, and when to self-host. llm ai go AI Startups and the GPT Wrapper Problem · 3 min Most 2023 AI startups are GPT wrappers the platform can absorb. What makes one defensible, from a former startup founder. ai startups strategy Building Semantic Search in Go with OpenAI and pgvector · 7 min Building semantic search with Go, OpenAI embeddings, and pgvector: structure-aware chunking, hybrid retrieval, an eval set, and the mistakes I made. ai llm go AI Code Review: What It Catches and What It Misses · 4 min Three months of AI-assisted code review on Go services: it catches unchecked errors and leaks, misses context, and only helps once you filter the noise. ai engineering developer-experience Fine-Tuning vs. Prompting: A Decision Framework · 4 min Most teams should exhaust prompting before they even think about fine-tuning. Here's how to decide which lever to pull. ai llm LangChain Is the New ORM: Convenient Until It Is Not · 4 min LangChain promises to simplify LLM development. Instead it adds abstraction layers you will fight against the moment your use case gets real. llm ai opinion RAG in Production: Patterns That Survive Real Traffic · 8 min RAG quality is retrieval quality. Chunking, hybrid search, query shaping, reranking, and evals for grounding LLMs in private data, with Go examples. llm ai go Vector Databases Explained: What They Are, When You Need One · 6 min What vector databases store, how similarity search and ANN indexes work, and when pgvector is enough versus a dedicated vector database. llm ai go Claude vs GPT-4: Two Weeks Using Both · 3 min Claude and GPT-4 both went public on March 14. How they compare on instructions, refusals, and reasoning, and why teams shouldn't marry one model. ai llm AI Safety Is Just Security Engineering With Extra Steps · 4 min For engineers shipping LLM features, AI safety is reliability, security, and accountability. A threat model and defenses from a security background. ai governance security My First Week Building with GPT-4 · 4 min My first week with GPT-4 after its March 14 launch: where it beats GPT-3.5, what it costs, and why I route requests instead of switching everything over. ai llm architecture Prompt Engineering Is Not Engineering · 3 min The term 'prompt engineering' oversells what is mostly clear writing. What works in a prompt, where teams waste time, and where the hard work is. ai llm opinion LLM Integration Patterns That Survive Production · 6 min LLM calls are slow, costly, and non-deterministic. Patterns for prompt versioning, structured output validation, RAG, tool guardrails, and fallbacks in Go. ai llm go AI in Production Is an Engineering Problem · 3 min ChatGPT reset expectations overnight. Shipping LLM features that hold up takes timeouts, validation, caching, cost tracking, and fallbacks. ai production engineering 2022 Year in Review: The Year the Music Stopped · 5 min A personal look back at 2022: building through the downturn, watching ChatGPT arrive, and what the year taught me about building things that last. year-in-review reflection ai Five Days With ChatGPT: First Impressions From an Engineer · 4 min Five days with ChatGPT as a working engineer: fast first drafts, confidently wrong answers, and why verifying the output now matters more than typing it. ai llm developer-experience GitHub Copilot After Six Months: Faster Drafts, More Review · 5 min Six months with Copilot in real projects. What it actually helps with, where it quietly makes things worse, and why the productivity claims are overblown. ai developer-experience productivity GitHub Copilot: First Impressions From a Go Developer · 4 min Early notes from GitHub Copilot's technical preview on Go code: strong on boilerplate and tests, wrong on domain logic, unresolved on licensing. developer-experience ai go MLOps Fundamentals: Fix the Basics Before Buying Tools · 4 min Most teams buying MLOps tools can't version their training data. Start with dataset versioning, a release checklist, and one monitoring signal. ai devops data ML in Production for Backend Engineers: It's Mostly Plumbing · 6 min What backend engineers need to know about ML in production: training-serving skew, model serving, deploys, and drift monitoring. ai backend engineering
All topics →