Writing / 2024
Claude 3.5 Sonnet Analysis: Cost, Coding, and Model Routing
Four days with Claude 3.5 Sonnet: coding quality, where Opus still wins, what Artifacts adds, and why it changes model routing for production.
Claude 3.5 Sonnet is the first mid-tier model I’d default to for most production workloads. It matches or beats GPT-4 on the coding tasks I care about, costs less, and Artifacts is useful for iteration. If you’re still routing everything to your most expensive model, run a side-by-side comparison.
Anthropic released Claude 3.5 Sonnet alongside a new Artifacts interface, and I’ve been running it against my usual workloads since launch day. This isn’t a benchmark review. Benchmarks tell you how a model performs on someone else’s problems. I care about how it performs on mine.
The positioning shift that matters
Every model provider has a lineup: cheap-and-fast at the bottom, expensive-and-smart at the top. The default instinct for production teams is to reach for the top tier because the cost of a bad output usually outweighs the cost of inference.
Claude 3.5 Sonnet challenges that instinct. Anthropic is explicitly positioning a mid-tier model as the default for serious work. It’s a pricing play and also a claim that the quality gap between tiers has narrowed enough that the mid-tier clears the bar for most real-world tasks. That is the same routing question I keep coming back to: which requests deserve the expensive path?
I’ve been testing this claim. Here is what stood out.
Coding: where it impressed me
I ran Sonnet through the types of coding tasks I deal with in my Go-heavy workflow:
Multi-file refactors. I asked it to rename a package, update all references, and adjust the tests. Sonnet got this right on the first try, including edge cases in test helper files that GPT-4 had missed when I ran the same task a month earlier.
Bug diagnosis from error traces. I pasted a stack trace from a concurrency bug in a Go service. Sonnet identified the race condition, explained why it manifested only under load, and proposed a fix using sync.Mutex that was correct and idiomatic. It didn’t suggest sync.Map when a plain mutex was the right call. That kind of judgment matters.
Documentation from code. I gave it a 200-line Go package and asked for a README. The output was usable with minor edits. It captured the intent, not just the function signatures.
These are the tasks where I spend real time. A model that handles them reliably at a lower price point changes how I think about routing.
Where Opus still wins
I found Sonnet’s limits in a few predictable places:
Long-form reasoning across large contexts. When I loaded a full design document (~15K tokens) and asked for a critique, Sonnet’s analysis was surface-level compared to Opus. It identified structural issues but missed a subtle consistency problem that Opus caught.
Ambiguous instructions. When the prompt is vague, Sonnet tends to make reasonable but sometimes wrong assumptions instead of asking for clarification. This is manageable (you just need more explicit prompts), but it means you can’t be lazy with your instructions.
Creative writing. Not my primary use case, but I noticed it. Sonnet’s prose is competent but flat. If you need compelling narrative or nuanced tone, Opus is still noticeably better.
Artifacts: more useful than I expected
I was skeptical of Artifacts when I saw the announcement. It looked like a UI gimmick. A few days of daily use changed my mind.
The core idea: when the model produces code, a document, or a visualization, it renders it in a separate panel instead of inline in chat. You iterate on it through the conversation, then copy or download the result. The model treats it as a persistent object in the conversation.
Where this is genuinely useful:
- Prototyping UI components. Ask for a React component, see it rendered, ask for changes, see the update. The feedback loop is fast.
- Drafting specs. The artifact is a living document that you refine through conversation. Much better than scrolling through a chat history to find the latest version.
- Quick visualizations. SVG diagrams, simple charts, Mermaid flowcharts. The inline render makes iteration practical.
It’s a real workflow improvement for anyone using an LLM for iterative creation.
How I’d evaluate this for your team
Don’t take my word for it. Run your own comparison . Here’s the approach I recommend:
- Pick 10-15 real tasks from your last two sprints. Skip toy problems and use actual things your team spent time on. Code reviews, bug fixes, documentation, data analysis.
- Run them through Sonnet and your current default model side by side. Same prompts, same context.
- Score on three dimensions: correctness, usefulness (did you use the output or throw it away), and time saved.
- Compare cost and latency. Sonnet should be meaningfully cheaper and faster. If the quality is comparable, the math is obvious.
Do this for a week, not an afternoon. First impressions are unreliable. You need enough data points to see the failure modes, not just the wins.
The model routing question
The bigger implication of Sonnet is routing, more than any Opus-versus-Sonnet swap.
Most teams use one model for everything. That was reasonable when the quality gap between tiers was large. Now that the gap is narrowing, a smarter approach is to route by task:
- Sonnet for coding, classification, extraction, structured output, and most day-to-day work.
- Opus for complex reasoning, nuanced analysis, and tasks where the cost of a wrong answer is high.
- Haiku for preprocessing, filtering, and high-volume tasks where speed matters more than depth.
Keep model identifiers and routing rules in config so you can shift traffic as models improve without redeploying.
Claude 3.5 Sonnet is the first mid-tier model where I stopped reaching for the top tier by default. Don’t switch blindly: test on your workloads, measure the quality gap, and route by task. For most teams, that will mean moving a significant chunk of traffic to Sonnet and saving the heavyweight model for the tasks that genuinely need it.