AI Economics: Marginal Costs Return

write-up · for knighthk in #systemcrafters · 2026-07-21 10:17 UTC

gist: AI models like Kimi K3 reintroduce marginal costs, reshaping the industry's economics and challenging misconceptions about "free" open-weight models.

Marginal costs are back: Running AI inference (e.g., generating tokens) incurs real costs, directly tied to revenue, unlike traditional software with near-zero marginal costs.
Misconception of "free" models: Open-weight models are not free to serve; inference costs like Kimi K3 ($3/$15 per million tokens) highlight ongoing expenses.
Economics of AI: COGS (cost of goods sold) is significant in AI, scaling with revenue, contrasting with software’s zero-marginal-cost model.
Nvidia’s role: GPUs act as "token factories," focusing on efficiency metrics like tokens-per-second, token cost, and power usage to optimize performance.
Industry shift: The rise of state-of-the-art models like Kimi K3 forces a reevaluation of AI’s value chain and long-term economic structure.

Commenters debate whether AI model "harnesses" truly create a competitive moat, with some arguing the model itself is the core value, not the surrounding tools. There’s notable discussion about the geopolitical implications of AI development, particularly concerns over U.S. dependence on China and calls for legislation to secure American interests in AI training and distillation. Disagreement emerges on the stickiness of AI tools like Claude Code and Codex, with some users finding them easily interchangeable while others highlight their enduring utility. Overall, the thread reflects a mix of technical skepticism, geopolitical anxiety, and differing user experiences.

See also