Quick take
Reliable agents are built, not prompted. Limit tools and steps. Validate every action at the boundary. Persist state so retries are safe. Design explicit recovery paths. Measure outcomes with evals, not vibes. If you want autonomy, earn it in increments with evidence and guardrails. This post includes the Go patterns I actually use.
I’ve been building agent systems in Go for the past year – across startups and enterprise teams. The same lesson keeps repeating: the model is the easy part. The hard part is everything around it. Tool validation. State management. Recovery paths. Observability. The boring infrastructure that turns “it works in a demo” into “it works at 3am when nobody is watching.”
Reliable agents are engineered, not prompted. Here’s how.
What “reliable” actually means
If you can’t write down the success criteria, you can’t make an agent reliable. “Handle this ticket” isn’t a spec. “Classify into one of five categories, draft a reply citing the relevant policy section, and escalate to a human if confidence is below 0.7” is a spec.
A reliable agent operates within known tools, limited steps, and explicit completion checks. It produces repeatable outcomes. It fails safely. Creativity and autonomy aren’t the goal. Predictability is.
Reliability is strongest where the task is structured: multi-step workflows with fixed tools, document extraction, data transformation with deterministic post-processing. It degrades as tasks become open-ended, long-running, or novel. That isn’t a temporary limitation. It’s a fundamental property of probabilistic systems.
The architecture that holds up
The reliable agent systems I build don’t look like a single prompt calling tools. They look like a small system with explicit responsibilities:
type Agent struct {
tools ToolRegistry
policy PolicyEnforcer
validator ActionValidator
state StateStore
supervisor Supervisor
maxSteps int
timeout time.Duration
}
type ToolRegistry struct {
tools map[string]Tool
}
type Tool struct {
Name string
Schema jsonschema.Schema
Execute func(ctx context.Context, args json.RawMessage) (json.RawMessage, error)
SideEffects bool
Idempotent bool
}
Every component has a clear job. The tool registry enforces schemas. The policy layer checks permissions before execution. The validator inspects arguments and output shape. The state store persists progress so retries don’t repeat side effects. The supervisor can stop, escalate, or hand off to a human.
You can implement this in a lightweight way, but the responsibilities need to exist somewhere. If they don’t, reliability will always be “mostly okay until it isn’t.”
Validation at the boundary
Agents fail in boring ways. Wrong parameters. Missing required fields. Calling the right tool at the wrong time. Repeating a write action. Getting stuck in a loop.
The fixes are also boring:
func (v *ActionValidator) Validate(action Action) error {
tool, ok := v.registry.Get(action.ToolName)
if !ok {
return fmt.Errorf("unknown tool: %s", action.ToolName)
}
if err := tool.Schema.Validate(action.Args); err != nil {
return fmt.Errorf("invalid args for %s: %w", action.ToolName, err)
}
if tool.SideEffects && !v.policy.Allowed(action) {
return fmt.Errorf("action %s denied by policy", action.ToolName)
}
return nil
}
Validate arguments at the boundary. Return structured errors. If a tool has side effects, check policy before execution. If a tool isn’t idempotent, check whether this exact action has already been executed in the current run.
This isn’t clever. It’s the same approach I use for any public API. Treat tools like APIs, enforce contracts, and the model has fewer ways to surprise you.
Idempotency and state
The nastiest agent bugs come from retries that repeat side effects. Duplicate tickets. Repeated refunds. Double-sends. The fix is the same as in any distributed system: make write operations idempotent.
func (s *StateStore) ExecuteOnce(ctx context.Context, stepID string, fn func() (json.RawMessage, error)) (json.RawMessage, error) {
if result, ok := s.Get(stepID); ok {
return result, nil // already executed, return cached result
}
result, err := fn()
if err != nil {
return nil, err
}
s.Set(stepID, result)
return result, nil
}
Every meaningful step gets a unique ID. Before executing, check if the step has already completed. If it has, return the cached result. This makes retries safe and recovery straightforward.
I learned this pattern while building cloud infrastructure at a previous startup, not AI systems. Same principles. Different surface area.
The supervisor loop
The supervisor is the most important piece. It enforces hard limits and decides what happens when things go wrong:
func (a *Agent) Run(ctx context.Context, task Task) (Result, error) {
ctx, cancel := context.WithTimeout(ctx, a.timeout)
defer cancel()
for step := 0; step < a.maxSteps; step++ {
action, err := a.planNextAction(ctx, task)
if err != nil {
return Result{}, fmt.Errorf("planning failed at step %d: %w", step, err)
}
if action.Type == ActionComplete {
return a.finalize(ctx, action)
}
if action.Type == ActionEscalate {
return a.escalateToHuman(ctx, task, action.Reason)
}
if err := a.validator.Validate(action); err != nil {
a.logValidationFailure(step, action, err)
continue // let the model try again with the error context
}
result, err := a.state.ExecuteOnce(ctx, action.StepID, func() (json.RawMessage, error) {
return a.tools.Execute(ctx, action)
})
if err != nil {
a.supervisor.OnFailure(ctx, step, action, err)
continue
}
a.appendResult(step, action, result)
}
return Result{}, fmt.Errorf("agent exceeded max steps (%d)", a.maxSteps)
}
Hard maximum on steps. Hard timeout. Explicit escalation path. Validation before every tool call. Idempotent execution. Structured logging at every decision point.
This isn’t a framework. It’s a pattern. Adapt it to your domain. The important thing is that these responsibilities exist in your system, however you implement them.
Observability
If you can’t see what the agent did, you can’t improve it. Log enough to answer practical questions:
- Tool name, step number, latency
- Success/failure codes and validation errors
- Argument hashes (not raw values for sensitive data)
- Completion status and reason for stopping
- Human handoff events
This data turns “the agent is flaky” into “the search tool fails 8% of the time when the query exceeds 200 characters.” The second statement is fixable. “Flaky” isn’t.
Where this falls apart
Open-ended creative work. Long-running autonomous loops with shifting context. Novel situations without prior examples. High-stakes decisions without human review.
These aren’t temporary limitations waiting for a better model. They are fundamental properties of probabilistic systems operating in complex environments. If your agent needs to handle these cases, the answer isn’t a better prompt. The answer is a human checkpoint.
The uncomfortable truth
Most agent reliability problems aren’t model problems. They are engineering problems. Wrong tool schemas. Missing validation. No idempotency. No timeouts. No escalation path. The model does something unexpected, and instead of being caught at the boundary, it cascades into a production issue.
Fix the engineering first. The model reliability improves as a consequence.
If you want autonomy, earn it in increments. With evidence. With guardrails. Not with optimistic prompts and hope.