Home / Concepts /Cost as Design Constraint

Cost as Design Constraint

When the model is fixed and cost is the binding constraint, pattern selection IS the design. This is where engineering judgment lives.

Definition

Most agent design writing assumes the way to scale is to use a better model. In production, the model is often fixed at the cheapest viable tier, and cost-per-call becomes a hard ceiling set by the business — not by the engineer. The design question moves from "which model" to "which patterns to compose so that this fixed model can carry this workload at this cost." Pattern selection under cost constraint is the actual engineering surface.

The problem it names

[TODO: 咖哥 voice — 真实场景。RegTech / 客服 / batch processing 这类生产场景里, "用更贵的模型" 不是选项 —— 业务方已经把 token budget 锁死。 这时候才看出工程师水平:同样的 Sonnet,有的人编排出可用系统,有的人卡死在 retry loop。 这一段建议你举一个具体例子,把 "cost-as-constraint" 从抽象变成画面。]

How it changes design decisions

Three shifts:

1. Reasoning depth becomes a budget, not a quality dial. Adding a Reflection loop is no longer "always good"; it's a 2-3× cost increase that has to earn its place.
2. Routing patterns become first-class. Complexity-Based Routing (R2) is not optional polish — it's the only way to keep average cost under ceiling when query distribution is skewed.
3. Caching and procedural memory become architectural. Skill Package (F2), Hierarchical Retention (M1), Failure Journals (M4) stop being optimizations and become load-bearing structure.

Lineage

• Database query planner under memory budget — same query has different optimal plans at different cache sizes
• Embedded systems — when RAM is fixed, algorithm choice IS the design
• SRE error budget — cost (of failure) as the binding constraint that shapes release strategy
• Cost-aware compilers — JIT optimization driven by profile data, not abstract preference

What it is NOT

• Not cost-cutting. Cost-cutting reduces an expense; cost-as-constraint accepts a ceiling and designs within it.
• Not "use the cheapest model." The model is given. The design is what you build with it.
• Not optimization. Optimization tunes a chosen design. Cost-as-constraint chooses the design.
• Not a Chinese-market peculiarity. Every production agent at scale eventually hits this; richer firms hit it later, but they hit it.

Where it's developed further

• The Pattern Selection Card methodology — encodes cost class as a first-stage filter
• Book treatment: Designing AI Agents (Manning, 2026) — Ch7 governance + Ch10 production harness
• Worked example: RegTech RAG case study (forthcoming as a public essay)

Cite as: Huang, J. (2026). Cost as Design Constraint. kage-ai.com/concepts/cost-as-design-constraint/