Move Fast, But Don’t Break the Magic

I have been thinking a lot about release velocity in AI products.

For the last two years, the industry has had a simple mantra: speed matters. The teams that are winning are learning faster, shipping faster, and getting real user feedback faster. In AI, where product behavior changes with models, prompts, tools, context, and defaults, speed has become a real competitive advantage.

But I think we are also starting to see the other side of that curve.

As a user, I have noticed that some products that felt magical a few months ago now occasionally feel inconsistent. Not broken. Not unusable. Just subtly worse. A coding assistant that crashes more often. A model that gives a shallower answer.

Anthropic’s recent Claude Code postmortem was a useful example. The issue was not simply that “the model got worse.” Anthropic traced the experience to product-level changes: reasoning defaults, a caching bug that dropped prior reasoning, and a system prompt change.

From the inside, those are separate systems. From the outside, they collapse into one user experience. The user does not experience the model, prompt, cache, tool layer, routing logic, and product surface as separate things. They experience one question: did this help me do the work?

That matters more now because these tools are no longer side experiments. They are becoming part of the daily workflow. People are relying on them to write code, debug, research, analyze, plan, draft, and make decisions. Once a tool becomes part of the work itself, the expectation changes. Novelty can tolerate rough edges. Workflow cannot.

This is where I think there is something to learn from Windows Update.

Microsoft does not push every meaningful Windows change to every user every day. There is a monthly security update rhythm, with preview releases for validation and out-of-band releases for critical issues. That cadence creates value in two ways: it gives users more stability, and it gives the company time to test, stage, validate, and communicate changes.

I am not arguing that AI companies should ship once a month. That would be too slow for this market. I would like my updates more frequently too.

But once a day may also be too fast for changes that affect quality, trust, or core workflows.

The question is not whether to move fast. The question is what kind of speed you are building.

Good speed creates learning. It gives users new capability without destabilizing the workflows they depend on. It is instrumented, staged, reversible, and intentional. Bad speed creates churn. It can optimize a local metric like cost, latency, or verbosity while quietly degrading the user’s actual outcome.

For AI products, the answer is not to slow down. It is to build a better release architecture. Deploy fast, but release deliberately.

Feature flags, canaries, staged rollouts, holdbacks, soak periods, and fast rollback paths are not process theater. They are how teams keep moving without making every user part of the test population.

Most importantly, define the embarrassment bar. Every team needs a threshold below which a change simply does not ship.

For AI products, that bar cannot only be crash rates and latency. It has to include output quality, instruction following, task completion, workflow continuity, user trust, and whether the product still feels as capable as it did yesterday.

Not every change needs the same process. A small UI improvement can move quickly. An invisible backend optimization can move quickly if it is reversible. A critical bug fix should move immediately.

But changes to reasoning behavior, system prompts, model routing, memory, coding agents, and user-facing defaults deserve a higher bar. In AI products, those are not implementation details. They are the product.

The companies that win will not be the ones that choose between speed and quality. They will be the ones that make quality part of their speed system.

The strongest teams will use evals, but they will not treat evals as the whole answer. They will dogfood the same experience users get in production. They will measure user-perceived quality, not just benchmark performance. And they will make behavior easy to roll back, not just code.

Speed is still a moat. But trust is also a moat. Trust is built when users feel the product getting better without becoming unstable. When yesterday’s workflow still works tomorrow. When new capability does not come at the cost of consistency.

Ship fast. Learn fast. But never ship below the embarrassment bar.

This article reflects my personal perspectives on product management, AI, and content discovery. It does not represent the official position of my employer or any affiliated organization.