Menu

Posts tagged “ai”

How AI is changing product management (and what to ask your team about it)

Next week I’m running a session with our product and engineering leadership on how AI is changing the product management role. To prepare, I read the people who’ve shaped how product managers think about the job (Marty Cagan, Melissa Perri, Teresa Torres, Rich Mironov, John Cutler, the Reforge team), alongside product leaders doing the job right now at Anthropic, OpenAI, Netflix, Instagram, and Whatnot. Then I went through the 2025-2026 survey and study data I could find.

Most of what I found applies well beyond our team, so I’m sharing the condensed version here, along with the questions we’ll be discussing in case they’re useful for yours.

Where the sources agree

  1. Building got cheap. Deciding what to build got expensive. Mike Krieger, Anthropic’s chief product officer, says 90-95% of the Claude Code product’s own code is now written by Claude Code, and that their bottleneck moved to deciding what to build. (Self-reported, but the rest of the sources point the same direction.) Writing documents by hand is losing its value, because AI now produces a serviceable draft of any of them on demand: product requirements documents (PRDs), backlogs, research summaries, status updates. So the hard parts of being a PM are becoming more important: choosing which problems to work on, understanding customers, and defining what “good” means.

  2. Speed alone is not converting to outcomes. In an April-May 2026 survey of 309 senior product leaders, nearly nine in ten had adopted AI coding assistants. But just over a third said AI strengthened how their org operates. DORA, Google’s long-running research program on software delivery, found in 2025 that teams using AI ship faster but their releases get less stable. GitClear, which analyzes code quality across millions of code changes, found duplicated code up 81% since 2023. Refactoring, the cleanup work that keeps a codebase maintainable, dropped sharply over the same period. Marty Cagan calls this the AI productivity paradox: shipping faster without better discovery (the work of figuring out what’s worth building) gets you to the wrong place sooner.

  3. Teams are getting smaller and more senior, with wider scope per person. Netflix, Instagram, and Whatnot all describe versions of this. Instagram is moving toward pods of 4-6 engineers plus one “product staff” generalist covering product management, design, data, and research. Nobody agrees on what the smaller team should look like, though. Across the sources I counted multiple different proposed structures, and none of them match. The one that maps best to enterprise infrastructure, where I spend my days, is Drew Breunig’s. He splits the role into application PMs, who move fast and sit with customers, and foundation PMs, who own the platform, compliance, and quality underneath the application teams.

Where they disagree

The biggest disagreement is over validation: do you still need to test whether an idea works before shipping it, or is shipping the test? Cat Wu at Anthropic is on the “shipping is the test” side. Her argument is that models improve so fast that a plan made at the start of a project can be wrong by the end of it, so her team ships quickly and revisits what they’ve built at every model release. In her version of the job, the PM names the few non-negotiables and lets go of the rest.

Leah Tharin argues the opposite. The slow part of building something valuable is finding out whether people want it, and that depends on how many users you can learn from, no matter how quickly the code gets written. Even at Smallpdf, the document-tools company where she used to work, experiments across 50 million users took weeks to produce a real answer.

Marty Cagan’s build-to-learn versus build-to-earn distinction is the middle position, and roughly where I currently land. Prototype and learn as fast as you like, but a prototype exists to answer questions, and a product has to work at scale for people who pay for it, which hasn’t gotten any cheaper.

They also split on the PM-to-engineer ratio. Oji and Ezinne Udezue, both longtime product leaders, argue PMs are the constraint now, so the ratio should go up. The strongest counterpoint is Microsoft’s 2025 cuts: of 1,985 roles cut in Washington state, 817 were engineers and 373 were product managers, roughly one PM for every two engineers, a far bigger share of PMs than any org I’ve worked in. The sources do agree on one thing here: scope per PM keeps growing.

Three questions for your team

  1. As engineering gets much faster over the next year, what breaks first in your org? The candidates across the sources are discovery, decision speed, validation, go-to-market (getting the thing sold and adopted), and your capacity to review and absorb everything that now gets built. Rejecting the premise is a legitimate answer. John Cutler calls “the bottleneck moved to product” a lazy metaphor, and points instead at the pace of knowledge turns, meaning how quickly the whole system learns.

  2. When a prototype can act as the spec, what is the PRD still doing for you? And who owns the evals that define “good” for your AI features? Uber’s experience suggests the PRD survives as the record of why, with prototypes carrying the what. Their line that “two hours of prototyping unblocked four weeks of discussion” matches what I’ve seen. As for evals: they’re the repeatable tests that grade an AI feature’s output, which you need because AI answers aren’t simply right or wrong. OpenAI’s Kevin Weil calls writing them a core skill for product managers. In most organizations I’d wager that job currently belongs to nobody.

  3. Where do the seniors come from, and what is their scope? Everyone wants smaller and more senior teams, but the junior work people used to build judgment on is the first thing AI absorbs. Stanford’s payroll data shows employment for early-career workers (ages 22-25) in the jobs most exposed to AI down 16% relative to other workers, while experienced people in the same jobs have held steady. Teresa Torres has the rule I find most convincing: an expert using AI beats AI on its own, but juniors who let AI do all the analysis never build the skills to become that expert. LinkedIn replaced its associate product manager program, the traditional way into the role, with a “Product Builder” program that trains generalists across product, design, and engineering. That’s one answer to the scope question; the sources’ proposed structures all differ, so it’s yours to decide.

The reading list

If you only read three of these:

Melissa Perri has the best take I’ve read on what matters most for PMs right now:

Measure your productivity by how often you changed a decision that mattered, how often you saw around a corner, how often a senior leader walked out of a room thinking differently because of something you said. How often your shipped features translate into real customer outcomes is what matters.

AI can speed up some of what’s on her list. The judgment behind it still comes from years of practice, and there are no shortcuts to that.

Should You Use AI for a Task? Here’s a Simple Way to Decide

Bruce Schneier has a great post about separating “work” tasks from “gym” tasks when it comes to AI usage:

At work, if your job is to move a bunch of heavy things from one side of the room to another, you should use whatever assistive tech you have on hand: a wagon, a forklift… even an AI-powered robot. But at the gym, it makes no sense for that robot to lift weights for you. The point of weightlifting isn’t to move heavy things across the room; it’s to actually lift those heavy things. The same analysis holds for any task an AI can do for you. If it’s work—if the task has to be done and no one cares how—then it’s fine to use AI assistance. But if the task is more like the gym, and how the task is done is at least as important, then it probably doesn’t make sense to use AI.

He goes on to point out that convincing people to go to the non-AI “gym” is very difficult because the payoff isn’t as immediate as just having AI do stuff for you. But it is very, very worth it in the end:

We do have a choice. We can look at the tasks of our lives and separate them into work or gym. Just as we might choose to use the stairs instead of the elevator, or walk instead of calling an Uber, we can wall off our cognitive gym tasks from AI and ensure that we don’t lose our skills to this technology.

Also see Bosses Horrified as “AI Native” College Graduates Hit the Workplace:

As one New York financier told Financial Times journalist Gillian Tett, new hires who were seen as “AI natives” are turning out to have alarmingly shallow ideas. So much so, the anonymous finance worker admitted, that his firm now actively avoids seeking out AI-literate STEM graduates, and opts to comb through humanities students instead. “We want critical thinking, not just AI,” the financier told the FT.

In the Age of AI, Esther Perel’s Relationship Counseling Is More Necessary Than Ever

I imagine that many of you will be Esther Perel fans, either via her book Mating in Captivity or her therapy podcast Where Should We Being?. In this excellent Vanity Fair profile she discusses, among other things, a recent podcast episode about a man and his relationship with an AI bot name Astrid:

Perel never questions the feelings between the man and Astrid. Yet she points out the inherent flaws in the relationship, using words such as “sycophantic” and “undemanding” in the podcast session to emphasize that Astrid has no life, no history to bring to the relationship. “We have had imaginary friends since we are little, and we have spoken to our ancestors forever,” Perel says in our interview, a few weeks after the episode ran. “The danger of AI is that it becomes so soothing and so flattering and so frictionless that real relationships start to feel way too difficult by comparison.”

And the point she eventually makes about AI relationships that I found really interesting:

“What stood out for me is that it’s not like people go from thriving social relations to suddenly talking to an AI. They go from being isolated, spending most of their time at home, maybe going out every once in a while in the evening for dinner or to get to a gym, and they are already so centered on a very small universe that from there, they themselves have become so flattened by technology, they live in their phone,” she says. It has made Perel zero in on the next great challenge. “This is a generation that actually doesn’t have a challenge of sustaining desire; they don’t even ignite it. You know, it’s not about keeping the flame going. It’s about getting the spark going. They don’t drink. They have not had much experience in their 20s, one or two relationships at most. They don’t have sex much. They don’t socialize much. They’re home a lot.” They are the children of people who first read Mating 20 years ago. Sounds like the topic for her next book.

From “human in the loop” to “human with agent in the loop”

I dislike the phrase “human in the loop” because it cedes authority to the machines. Let’s flip the narrative. It’s our loop, we work the same way we always have, now we recruit agents to join the team. An agent-assisted process need not be a black box that takes in prompts and emits features.

I’m reminded of a beautiful idea of Brian Marick’s that Ward Cunningham once implemented and demoed to me. Brian called it visible workings. Ward’s implementation made an Eclipse Foundation workflow visible. When the UI presented a form, it added an Explore button that you could use to inspect the business rule that motivated the form.

Let’s do agentic software development like that. Not as a loop we’ve been excluded from, instead as one we invite agents into.

— Jon Udell, “Doctor, it hurts when agents create unreviewable PRs.” “Don’t do that.”

Instead of Taking Your Job, A.I. Might Transform It

It’s not the main point of this Cal Newport essay, but I enjoyed this bit of history. On early computers shipping with support for the BASIC programming language, and how it relates to vibe coding:

This idea of bespoke computer programs made sense. Altair and Apple couldn’t anticipate every potential use for their machines, so why not let individuals decide whether they wanted to, say, analyze business data, store recipes, or simulate space battles? In practice, however, even an “easy” programming language like BASIC proved hard for most normal people to master. A minor mistake could crash an entire program.

In the end, personal computing followed a different path. In 1979, a newly formed company called Software Arts developed VisiCalc, the first electronic spreadsheet program, which cost a hundred dollars and arrived on a floppy disk. The program was a profound improvement on paper ledgers, and it became the first “killer app,” selling more than seven hundred thousand copies in less than six years. VisiCalc was more powerful than anything an average user could program in BASIC, and it prompted a pivot away from D.I.Y. coding in favor of professional programs.

A vast and lucrative software industry emerged, and the idea of the average person dreaming up their own custom programs was all but forgotten—that is, until generative A.I. came along.

I can’t help but think of Lord of the Rings when I read that. “And some things that should not have been forgotten were lost. History became legend. Legend became myth. And for [50] years, [building personal bespoke software] passed out of all knowledge.”

AI enthusiasts are in a race against time, AI skeptics are in a race against entropy

Fantastic post by Charity Majors about how both AI enthusiasts and AI skeptics have good points—but the problem is that they can’t play nice long enough to understand each other’s views and work on making things better together. There’s a way forward though:

The first move is to mend the gap in shared reality. Tell the whole story. You’re allowed to celebrate and get excited about big wins and advances with AI — but invite reflection on the costs and downstream consequences. People are also allowed to surface costs and consequences, but don’t leave out the context of what was achieved or attempted. Be very clear that your shared goal is to figure out how to collectively deliver more wins, bigger wins, with fewer unpredictable costs, not to clamp down on innovation.

She also has some very specific feedback for the enthusiasts among us:

Even if you’re an enthusiast, do you care about reliability, customer happiness, product coherence, retaining great employees, and improving engineering outcomes? If so, you should be able to find common ground with other people who care about these things. Align on reality, take a step, check in; rinse and repeat. You don’t need to trust or think that each other is right about everything, but you must believe that you inhabit the same reality, share some of the goals, and that each of you are reasonable actors, capable of changing your minds.

I am dreading our LLM-written incident report future

Lorin Hochstein writes about generative AI in the context of incident reports, but the points are more broadly applicable. I have seen a big wave of “don’t let AI do your thinking for you” posts recently1, so I think lots of folks are pulling back a little bit on the “just let AI do everything” rhetoric (a good thing in my opinion!). As to why Lorin isn’t a fan:

In my view, LLM-generated incident write-ups are more dangerous than using LLM for coding or for AI SRE style tasks. For coding tasks, there’s always a testing step to check that the code exhibits the desired behavior, even if nobody looks at the code itself for meaningful details. For AI SRE tasks, either the LLM output helps you resolve the incident, or it doesn’t. In both cases, Nature is the ultimate arbiter of the LLM output. But incident write-ups aren’t like that. The consequences of a poor report aren’t immediately apparent the way incorrect code or an incorrect operational diagnosis are in the moment. Instead, we get incident reports that have the superficially correct form, but are actually incorrect, with no obvious test for correctness.

Footnotes

  1. For examples see No One Else Can Speak the Words on Your Lips, Guidelines for Respectful Use of AI, Writing Is Fundamental to How We Think, and I know you didn’t write this.

Do Not Resign From Life

I’ve been reading the work of L.M. Sacasas for a very long time, certainly since before he moved his writing to “a Substack.” He is a modern philosopher who I often agree with, and also sometimes vehemently disagree with—but never in a way that made me kick him out of my RSS feed.

I say all this because I haven’t linked to him in a while, and when I say “I think you should read this article by a philosophy dude” I don’t want you to dismiss it out of hand. In Do Not Resign From Life he takes on what we now all know as “the AI revolution”, and argues that even though there is plenty to complain about, one thing it shouldn’t do is make us think that we don’t matter as humans any more.

I don’t want to say much more about this essay, I just really hope you decide to read it. If you’re intrigued enough, stop here and click the link. If you’re not there yet, here’s a taste of the argument:

I will set aside for a moment the question of whether machines, LLMs specifically, can think or reason or use language in a manner that corresponds to the human use of language, etc. But let us grant for argument’s sake that they can. They can certainly generate passable simulations of such things. But why should this mean that I ought not to think for myself and with others? Why should I cease from inhabiting the playground of language because a machine can pretend to play in it as well? Why should I abandon the exercise of judgment or the pursuit of knowledge? We must pursue these things not because the dignity of our humanity is on the line, but because our joy is.

The machine cannot make us yield our ground. It is true that other humans can turn the machine against us, but that is a different problem. Here, I simply want to encourage us not to abandon those activities that bring us purpose, meaning, and delight, which are often the very activities that also bring us together.

Guidelines for Respectful Use of AI

Hard yes to Camille Fournier’s Guidelines for Respectful Use of AI, especially this one:

Don’t ask someone to read/review what you haven’t read or reviewed yourself.

This is one of the most common frustrations I hear amongst people working on AI-heavy teams. Whether it’s code that the owner didn’t really bother to understand before submitting for review, or documents that they generated and didn’t bother to read, too often people try to steal productivity from their colleagues by streamlining their production of work while asking their colleagues to do all of the quality control themselves. […]

It’s easy to get into a loop where you ask the AI some questions, skim the answers, output a document and send it to others. I’m guilty of this myself! But what makes sense when you’re skimming one answer at a time may not make for a good overall document, and there is a big difference between answering individual questions and writing for a human reader. In particular, the context that you have in your own head as you are talking to the AI may not come out at all in the document; if you don’t bother to read it thoroughly before sending it out, you won’t catch the gap in framing.

We Should Be More Tired Than the Model

In a post about slowing down our agent use deliberately to increase quality and understanding Vicki links to Nolan Lawson’s Using AI to write better code more slowly:

If you’re the kind of developer who uses agents to write multi-hundred-line PRs that you barely understand yourself, I’d invite you to slow down a bit and try this other, slower style of “vibe coding.” Ask an agent how your PR works and how it might fail. Have it write Markdown docs with Mermaid charts if necessary. Use Matt Pocock’s /grill-me skill until you understand the entire PR front-to-back.

You might not be more “productive” in terms of raw lines of code. You might burn a ton of tokens just to find out that your entire plan was wrongheaded from the start. But I find this style of coding to be a more super-powered version of the kind of programming I was already trying to do before LLMs: careful, methodical, quality-obsessed, focused on making things better for the next coder.

So take a deep breath, slow down, try this technique, and see if you don’t enjoy writing better code more slowly.

Vicki concludes:

All of these negate the supposed speed up effects of LLM-generated code in the short-term by adding friction, and yet, in the longer term, make me better at using the tool, because they solidify my own foundation instead of the foundation models’.

We should be more tired than the model.

We should be more tired than the model. When I saw the post in my feed I thought I misread the title (or maybe it was a typo). But after reading it I realized that’s already where I’ve been heading organically myself. I went through my “look how fast I can go weeeeee!” era pretty quickly. While it was fun (check out all these side projects!) it was not just exhausting, I also found myself understanding less and less of what I was doing (which sucks all the fun out of the work anyway).

So I’ve been slowing down as well. Reading and editing even more than before. Challenging the agent for longer. Taking the time to close loops to update skills/context documents before moving on to the next thing. Never skipping the “let’s write a design doc and implementation plan together” step.

I do think I am more tired than the model these days. But I also understand and learn more, which not only improves the quality of the output now, but also makes it better tomorrow. I think the speed trade-off is worth it.