Many people game the way they measure success to maintain appearances and feel better about themselves.

It tends to backfire, especially in the long run, and the most dangerous flavors are a lack of awareness and lying to yourself. You can end up walking towards a fake north star that leads you nowhere.

A few obvious examples, definitely not a comprehensive list:

• Startup founders obsessing over signups and clicks instead of metrics like monthly recurring revenue (MRR) and customer retention rate (CRR).

• Managers taking pride in the number of people reporting to them.

• Product people boasting about how many new features they squeezed in this month.

Those metrics can tell us something about how skilled somebody is at certain things that don’t directly move the needle. They can also be proxies for meaningful outcomes, contingent on other, more important conditions being true. But on their own, they don’t tell us whether we’re actually winning.

For instance - the number of employees might mean something positive if revenue per employee is high and employees are happy and stay with the company for a long time. But it could mean a disaster if revenue per employee is alarmingly low and employees keep churning.

The scary thing is that we can do this without even being aware of it. Especially at the beginning of our journey, we need to deliberately find ways to become aware of the right things to measure and train ourselves to actually measure them. To optimize for signal over noise - what actually drives meaningful outcomes that move the needle.

A New Vanity Metric - AI Agent Count

In software development, there’s a new vanity metric - the number of AI agents running in parallel. The more agents you can spin up at once, the more of a genius you are. Or so the story goes.

As with other vanity metrics, this number alone doesn’t tell us whether we’ve achieved meaningful outcomes, and it’s pretty easy to cheat with it to make yourself sound more impressive than you actually are.

If someone uses 1,000 agents in parallel to write a new feature end-to-end in a few hours, are they a genius? Managing that many agents at once must require extraordinary amounts of intellectual effort.

Naturally, that statement in itself doesn’t tell us much - it’s mostly noise.

To someone who hasn’t used AI agents to code extensively, or at all, that number might suggest they’re managing a different context and set of hard problems for each agent. And developers might not be in a hurry to correct that impression.

To be fair, if you actually are juggling 1,000 different contexts in your head, you probably are a genius. Of course, that’s rarely the case. It’s pretty trivial to launch many agents that all relate to the same problem. The number of agents doesn’t equal the number of distinct contexts and problems managed in your brain.

To illustrate this point - with 1 quick prompt, one could fan out agents to generate 1,000 blog posts in parallel, with each agent’s subagents doing research, crafting the post, and checking its quality. And voilà, they’re using 1,000s of agents.

In practice, it doesn’t mean all of those run concurrently. There are often concurrency limits, so some agents end up queued. But you get the point.

The number of agents running in parallel joins the good old metrics like lines of code, commits, and PRs. In the right environment, they can mean something. On their own, not really.

The actual signal usually (always?) goes back to how something affected the business - either by measuring ROI directly or by measuring things that indirectly translate into sustainably running a good software business (e.g. code maintainability).

Unsupervised Agents

There’s this new hype around developers and tech influencers claiming they’re using 100s or 1,000s of agents to push PRs fully unsupervised. To be fair, those claims are often about producing correct code (as in, it compiles and works), rather than good code (maintainable, scalable, performant…).

Out-of-the-box configurations like Claude Code with Fable on Max & Ultra + gstack seem powerful, yet it seems one still has to keep their agents on track and help them iterate on problems. Or worse, solve the problem themselves and leave the agents to do only the dirty work.

Some people go deep into optimizing their own agents - e.g. by writing a very detailed set of skills. There’s definitely value in that, and you get full control over personalization. The better the agents are tailored to your preferences, the more you can leave them running unsupervised around your codebase, at least in theory.

That said, it seems like you mostly end up with personal flavors of gstack and pstack. Which is fine, but I’m not sure it’s worth the effort, given how quickly the best ideas tend to make their way into out-of-the-box solutions.

There’s probably a good 80/20 here: use existing tools for the general workflow, and focus your customization on your unique development preferences and the needs of each project. Some personal preferences will simply never be common enough to make it into off-the-shelf solutions.

I think there can be a healthy balance between code quality and development speed - more code, faster, with a tolerable percentage of AI slop. But above a certain percentage of slop and below a certain quality threshold, the maintainability tradeoff doesn’t seem worth it, though that’s likely subjective.

For now, the quality of the AI slop is still highly inconsistent, and I personally remain uncomfortable trusting AI agents to roam around unsupervised, pushing PRs for me.

I imagine that over time we’ll be able to let swarms of AI agents do more and more unsupervised - at least, that seems to be the current trend. But it doesn’t feel like they’re good enough to be left on their own just yet.

Where AI coding still falls short

AI is an umbrella term that can include potential, unpredictable breakthroughs. Since nobody knows if, when, or how those will happen, let’s focus on agentic LLM systems to keep things relatively grounded.

I went from being dismissive of them - unable to leave Claude alone unsupervised - to trusting it with more and more code.

The biggest jump came from feedback loops: inspect screenshots, run tests, see what broke, fix it, repeat.

The problem is that AI can produce something that works perfectly well while still being much less maintainable or scalable than it could’ve been.

It’s like a coding interview where there are 20 ways to solve the same problem. The naive solution might work for the immediate use case, but depending on seniority, that’s not enough. It might fall apart under scale, perform badly, duplicate existing abstractions, or just leave behind a system that’s unnecessarily hard to change.

I think the next big jump could be better feedback loops for code quality, not just correctness. Some of that already exists through linters, architectural rules, review agents and evals. But judging whether an abstraction is right for the system is much less deterministic.

I still don’t know if AI could entirely replace developers. My gut right now says no. At least not under the current paradigm.

It’s basically like harnessing a fleet of productive juniors with uneven judgment. Sometimes it’ll recreate hundreds or thousands of lines of plumbing instead of using a higher-level abstraction that gets the same thing done in 2-3 lines.

Those mistakes compound fast. Without someone who understands the broader system and roadmap guiding it, the code can become a shitshow pretty quickly, with the AI chasing its own tail fixing one bug after another in what turns into a never-ending game of whack-a-mole.

The tradeoff

AI coding presents a tradeoff between development speed and code quality.

For quick prototyping and pre-PMF iteration, I think that tradeoff is amazing. Moving fast matters more, and some technical debt is a rational price to pay.

Having bad code post-PMF seems like the right problem to have, to the dismay of us developers who reluctantly have to clean up the mess once the most chaotic part of iterating on the product is over.

But no-coders building scalable, maintainable, polished products entirely with AI still seems like a myth to me.

For critical parts of mature or enterprise systems, I’m much less convinced. I’d still let AI write straightforward UI and CRUD after the system is designed, but I’d give it much less freedom over architecture.

What if the AI bubble pops?

I also worried this whole thing might disappear when the AI bubble pops. There’s enormous capital going into frontier AI, and labs are spending huge amounts on training and infrastructure.

But inference itself can still have healthy unit economics once the model exists. So even if the current frontier-model race turns out to be a bubble, that doesn’t mean LLM inference disappears with it.

So I don’t think LLMs are going away.

What if AI actually replaces developers?

Let’s assume, though, that AI eventually entirely replaces software developers. What then?

Let’s put aside the utopian dream of everyone having abundant resources without needing to work. Even if that eventually happens, I doubt it’ll happen anytime soon.

Assuming people still have to work, I think the most sensible response is to recursively find areas AI still can’t do well and learn how to do them well.

That could mean going deeper into tech - systems thinking, distributed systems, deep domain knowledge, and solving hard or novel problems where there isn’t an obvious existing pattern to follow.

It could mean getting better at management, leadership and recruitment - while we still need humans to guide AI.

Or it could mean moving laterally into areas that depend much more heavily on humans dealing with other humans - like product and sales.

The boundary will keep moving. So should we.