AI Technical Debt

AI technical debt is harder to see than ordinary code debt. It hides in prompts nobody owns, evaluations nobody runs, stale embeddings, unpinned model behavior, and workflows where every small change feels risky.

This hub connects the older technical debt writing with the newer AI-specific debt pattern: invisible system behavior that keeps returning 200 OK while quality quietly degrades.

Start Here

• AI Technical Debt: Prompts, Evals, and Pipelines Nobody Owns is the current canonical piece on prompt debt, eval debt, pipeline debt, and architecture debt.
• AI Technical Debt: Prompts, Models, and Evals Nobody Tracks explains the same failure mode from an earlier production AI perspective.
• The True Cost of Technical Debt, and How to Measure It gives the non-AI baseline: measure the pain before arguing about priority.

Common Debt Types

Prompt Debt

Prompts become legacy code when they are copied from demos, edited without review, and never tied to expected behavior.

Evaluation Debt

Teams without evals argue about quality using anecdotes. That makes every model, prompt, or retrieval change risky.

Data Pipeline Debt

Stale embeddings, missing documents, drifting labels, and weak source tracking create confident wrong answers.

Architecture Debt

Provider-specific logic and tool calls buried in application code make upgrades feel like surgery.

How to Pay It Down

Start with the highest-leverage stabilizers:

1. Put prompts and retrieval configs in version control.
2. Add a small evaluation set for the highest-risk workflow.
3. Track model versions, prompt versions, and data freshness together.
4. Add cost attribution by feature or workflow.
5. Decouple model access from product logic before switching vendors.

Supporting Reads

• AI Evaluation and Production Governance: A Maturity Model
• RAG Retrieval in Go: Hybrid Search, Chunking, Reranking
• How We Track and Prioritize Tech Debt at a Fintech Startup
• Technical Debt Triage: A Simple Prioritization Framework

Related Hubs

• AI Operating Systems
• Quality
• Architecture

References

• NIST AI Risk Management Framework
• OWASP Top 10 for LLM Applications
• The Twelve-Factor App