Menu

Metrics to check when evaluating a company as a job seeker

There’s some really good advice for job seekers in Carilu Dietrich’s post 10 Most Important Metrics For Evaluating a Company:

Net Dollar Retention (NDR) is one of the best ways to see if a tech company is healthy and growing. It measures the expansion or contraction of a company’s existing customer revenue over a given period. NDR takes into account the revenue lost from churn and increased revenue from upselling, cross-selling, and renewals from existing cusomers.

I would also add Gross Revenue Retention to that list. It describes how much revenue a company keeps from its existing customers over a specific period, not including any new revenue from new customers. So, if a company started the year with $100 from its existing customers and ended the year with $90, even if they lost some customers but upsold others, the gross revenue retention would be 90%. It helps guage how good a company is at keeping its current customers happy and continuing to spend money, without taking into account any new business they’ve gained.

How generative AI might change the product profession in the future

I promise this isn’t going to become an AI blog, but Marty Cagan’s latest on Preparing For The Future has some solid points on how generative AI might change our profession. Here he talks about the impact on QA:

The new generation of AI-based test automation tools has the promise to revolutionize our approach to ensuring the product is behaving properly. On the other hand, our current approach to quality is largely based on deterministic products.  This means that given a set of inputs we can predict what the appropriate output should be, and we can count on that being true indefinitely.  

Yet for many new products built on generative AI, our products are no longer deterministic, but rather probabilistic.  We can no longer count on the same inputs generating the same outputs.  For many contexts, this is not a problem.  But for other contexts, especially when safety is involved, this will require different approaches to ensure appropriate behavior.

Also, yes please:

For those product managers of empowered product teams, the time spent creating artifacts such as written narratives, roadmaps, PRD’s, and acceptance criteria, as just a few examples, should be significantly assisted by the new generation of tools, some of which are already starting to appear. Even at the very rudimentary level, if the new generation of tools can significantly reduce the time a product manager (or engineer) spends dealing with tools like Jira, that will be a substantial win.

Evaluating AI product opportunities by plotting them on a “Survival Curve”

Aniket Deosthali (Head of Product for Conversational Commerce at Walmart) provides a great framework for evaluating AI product opportunities in How to Build AI Products People Want:

The most efficient way to evaluate AI opportunities and unlock the advantages of AI is by using the Consideration x Context framework. Let’s start with some baseline definitions.

Y-Axis = Consideration: The amount of effort required to make a decision.

The more thought you put into a decision, the higher consideration it is. For example, choosing a dish detergent is “low consideration” for most shoppers, compared to buying a car which is “high consideration.” Consideration can be represented as a function of the number of compelling alternatives and the stakes – users’ tolerance for errors, for example.

X-Axis = Context: The volume of abstract concepts AI needs to know. 

Context refers to how many abstract concepts a model needs to know in order to provide a useful response. Does it only need to understand a small batch of data points (like a product catalog), or does it need to understand the entire internet (like ChatGPT)?

He goes on to explain how to plot different solutions on a “Survival Curve.” If you work on AI products this is one of the rare actually helpful articles for PMs in the area of AI and LLMs.

Experimentation in the real world: Southwest Airlines

The post 7 innovations that Southwest is testing to improve its crucial turn times is a great real-world example of experimentation in product (make sure your ad blockers are charged for this one, it’s published on The Points Guy…).

Zach Griff goes over several ideas Southwest Airlines are trying to improve the time between when a flight arrives and leaves again. For instance, for when you’re queuing on the jet bridge:

The first is the installation of Bluetooth speakers in the jet bridge, which play (royalty-free) disco, electronic dance music, hip-hop and kids music during boarding and deplaning. Listening to music at a high beats-per-minute rate is scientifically proven to get people moving faster and more efficiently, according to McCartan, which is exactly what Southwest wants during one of the most critical phases of the turn.

There are lots of learnings for PMs sprinkled throughout the post.

How to communicate problems effectively to your manager

The Reforge team has a long post on How To Master the Art of Managing Up. I find this aspect especially important:

Those who have mastered managing up will package problems in a way that takes their managers’ constraints into account,  including time, lack of resources, or competing priorities. 

Your approach to packaging and communicating difficult situations can make the difference between managing up effectively and just causing more chaos for your manager.

They go on to provide some very good, practical tips for how to package problems effectively.

Creating from a deeper place

There’s a lot going on in John Warner’s Speed and Efficiency are not Human Values. It’s primarily a reflection on generative AI tools in the context of being a published author—and well worth reading.

But the reason I am linking to it here is because it gives you an excuse to watch (or re-watch!) what John calls “the greatest guitar solo ever captured on a recording” (he is 100% correct). Here is Prince at the Rock and Roll Hall of Fame induction ceremony in the year both he and George Harrison (posthumously) were enshrined:

Here’s how John describes the solo in his post:

Prince was obviously a highly skilled guitarist capable of blazing speed on the fretboard (like the “Flight of the Bumblebee” guy) and indeed there’s a couple of spots where he just rips through some rapid note runs, but it’s also intensely musical, totally its own thing, while also managing to reference aspects of the solo from the original version (performed by Eric Clapton). […]

A great guitar solo is not about how fast you can play, or your degree of technical skill. It comes from a deeper place.

I know you’re going to roll your eyes, but seriously, the solo and that quote—it comes from a deeper place—inspires me to think a little bit more about the feel of the products we make, and a little bit less about the ”correctness” of fitting a specific mold.

How much product data is enough?

I really like this list of heuristics by Itamar Gilad for how much data you should have before launching product features/improvements:

  • Never launch anything solely based on opinions.
  • It’s ok to release minor tweaks based on assessment.
  • Having supporting data (without testing) is enough to launch only very small, low-risk, easy-to-revert changes.
  • Everything else should be validated through tests and experiments. However there are various levels of testing to choose from with different associated costs and confidence levels.  
  • How much validation you need depends on: a) the cost of building the idea in full, b) how risky it is, and c) your risk tolerance. 

Conf Meter Quads scaled

It’s also the first time I’ve come across his confidence calculator, which looks like a really useful tool (email-gated, though…).

Why productivity might be falling in organizations

Here’s a good theory by Bruce Daisley about the real reason why productivity might be falling in organizations:

If you want to understand why productivity is falling, we need to look first at high levels of employee turnover. If we want to solve productivity issues the first step needs to be to lower the resignation rate.

We all know well when people quit their jobs a period of unproductivity commences: bosses and colleagues need to cover the work of the person leaving, the recruitment process takes unproductive attention and new starters take months to ramp up. As Ton says, ‘high employee turnover is ruinous for productivity’.

Blaming “low productivity” on the rise of remote work—like some publications are trying to do instead—seems pretty lazy.

More

  1. 1
  2. ...
  3. 21
  4. 22
  5. 23
  6. 24
  7. 25
  8. ...
  9. 201