Menu

How we got here (it’s not a “root cause”, it’s the system)

Lorin Hochstein shares a characteristically solid systems-thinking take in CrowdStrike: how did we get here?:

Systems reach the current state that they’re in because, in the past, people within the system made rational decisions based on the information they had at the time, and the constraints that they were operating under. The only way to understand how incidents happen is to try and reconstruct the path that the system took to get here, and that means trying to as best as you can to recreate the context that people were operating under when they made those decisions.

The “no root cause” concept is something I’ve been thinking about a lot as I’m working on a particularly complex project at work. Somehow we constantly forget that things usually are the way they are not because of a single “mistake”, but because of a the culmination of a bunch of legitimate reasons.

Systems get the way they are because of decisions made in good faith based on the data available at the time. And the worst thing you can do as a new person coming in to improve things is to hunt for a single “root cause” to fix. That’s just not how software (or people!) work. So take the time to understand Chesterson’s fence. Go ahead and draw boxes and arrows until no one disagrees any more about how the system works. And then figure out which parts can be improved, and in which order.


PS. Also see How Complex Systems Fail:

Because overt failure requires multiple faults, there is no isolated ‘cause’ of an accident. There are multiple contributors to accidents. Each of these is necessarily insufficient in itself to create an accident. Only jointly are these causes sufficient to create an accident. Indeed, it is the linking of these causes together that creates the circumstances required for the accident. Thus, no isolation of the ‘root cause’ of an accident is possible. The evaluations based on such reasoning as ‘root cause’ do not reflect a technical understanding of the nature of failure but rather the social, cultural need to blame specific, localized forces or events for outcomes.

High-reaching informality and the difference a good manager makes

Neil Perkin tackles some important leadership points in High-reaching informality (emphasis mine):

Overly formal environments are more likely to be low in psychological safety, and less likely to encourage people to speak up, contribute their ideas, and challenge assumptions or norms. Informality helps to break down barriers, reduces the toxicity and influence of internal politics, helps a team to get the best ideas and to be adaptive in delivery to arrive at better outcomes. […]

It’s useful to think of psychological safety as the bringing together of mutual trust and respect, and also comfort with dissent. Overly or inappropriately formal environments reduce trust down to a transactional relationship: I asked you to do this, so did you do it? Whilst dependability is important, trust is also built on competence, confidence, integrity, and empathy.

This relates to something that’s been on my mind quite a bit over the past few weeks: the question of what it means to be “boss”. The “transactional relationship” that Neil describes above is one I will always avoid with all my heart with my teams.

I was lucky early in my career. When I first joined eBay I had a fantastic manager. He was empathetic and people-first, and yet strongly business-driven. He coached me, corrected me, taught me how to speak the language of business, challenged me… but above all, it felt like he was on my side. I had someone in my corner, someone rooting for me, someone to help me navigate a gigantic organization while I really felt like I had no idea what I was doing. He didn’t react negatively to bad news or frustrated questions, he simply said, “tell me more about that.” I will be forever grateful to Christian for the way he set me up for long-term career success.

20 years later I know just how lucky I was. From my conversations with teams and candidates over the past year or so it has become clear to me that very few people have the privilege of working with a good manager early in their career—someone who is supportive yet determined for us to do the best work of our lives, together. This seems particularly acute in the middle management layer1, where organizations spend very little time on training new leaders. From The Strengths, Weaknesses and Blind Spots of Managers:

Worldwide, the cost of poor management and lost productivity from not engaged or actively disengaged employees is $8.8 trillion, or 9% of global GDP. Changing how people are managed is perhaps the easiest way to boost productivity within organizations.

Yet, the majority of managers receive little feedback on how effectively they manage their team. Less than half of U.S. employees (42%) report having the opportunity to formally provide feedback to their manager, and fewer than one in four (24%) have formally rated their manager’s performance.

I’m not sure what the solution is, but I do think this is an important problem to call out and watch out for. Do you have transactional leaders in your company, or relational ones? The transactional leaders might initially appear to be more effective, but everything I’ve learned up to this point in my career has shown me that those benefits are short-lived, and almost always end up with disengaged, resentful team members. As leaders we are not here to tell people what to do. We are here to provide strategic direction and context, and coach our people—as part of the same team—on how to make the business and themselves successful for the long haul.

PS. If you’re looking for a book that covers these topics really well, give Good Authority: How to Become the Leader Your Team Is Waiting for by Jonathan Raymond a go.

  1. See Director means something for a devastating take on this ↩︎

Grow down

I love this reflection about personal resilience from Mandy Brown:

That is, we grow not only up—not only skyward—but down, into the roots, back to that from which we came and to which we will, one day, return. We become, in time, more rooted and resilient, more capable of surviving the storm, less easily shaken away from ourselves by idle wind or rain. When I think about growing down instead of up, I think about becoming centered, about knowing what work is ours to do (and, critically, what work is not), about a slow, steady power rather than a rash and inconstant one. After all, as anyone who’s ever lived among city trees can tell you, neither brick nor concrete nor iron can stop a root as it seeks out water. We should be as steady in our search for that which nurtures our own lives.

You are not your team’s umbrella

Two recent articles about engineering management stuck with me because they complement each other well to articulate a leadership philosophy I’ve held for a long time, but haven’t been able to put into words effectively.

First, in The Disappointment Frontier James Stanier expands on a controversial quote: “leadership is disappointing people at a rate they can absorb.” He talks about how keeping teams in a “bubble of protection” (where you act as the “umbrella” for bad news) never ends well. And even though managing disappointment is a core leadership skill, it is typically overlooked when training new managers. It’s worth reading the whole article, but here is a key point:

Managing a team of any size—from a handful of people to a whole company—is a continual balance between trying to empower people to achieve what they want (interesting projects, plentiful opportunities, a plethora of pay increases and promotions) whilst navigating a conflicting reality that doesn’t always want to give it to them. As such, there is a lot of truth to the quote at the beginning of the article. Leadership is about disappointing people at a rate they can absorb. In fact, I think that I’ve likely spent far more time on mitigation of disappointment or on managing expectations than I have on being motivational in my management career.

James goes on to give some really good advice on how to manage disappointment well. The second article is Will Larson’s Unexpected Anti-Patterns for Engineering Leaders, in which he also discusses the dangers of the “umbrella” approach:

The more senior the teams I’ve worked with, the more I’ve come to believe that protecting your team from reality just makes things worse for them in the long term. Larson now tries to throw his team into the gory details much more quickly. “In the past, I used to think I was energizing my team by sparing them the details. But now it feels like lying to them,” he says. On a tactical level, Larson adjusted this habit by buffering less information. “That means even if it’s disappointing for folks, I’d rather them process news bit by bit, rather than deal with a huge ocean of mess all in one moment.”

Obviously you are not going to tell your team every little thing that happens in your day and your meetings. But the idea that we should shield our teams from the realities of an organization is fundamentally flawed and ultimately does a huge disservice to people. If we hide the context of our decisions or guidance we lose the trust of our team—and once that happens, all is lost. This approach also stunts growth. If we don’t prepare people for how organizations work, their eventual exposure to reality will be a huge shock and we would’ve lost out on the opportunity to coach them slowly over time.

I once received some performance review feedback that I am “too transparent” with my teams, and I need to dial that down a bit. I respectfully declined. I don’t know of any other way to be—both as a human and as a manager. I also firmly believe I would be a worse people manager if I adopted that feedback.

That said, transparency is a skill. One I’ve cultivated for many years, and will likely never fully master. How and when you say things matter, and it takes time to learn those nuances. The above articles give some great advice on how to start that journey with your teams. So I encourage you to give transparency a try. I think you’ll find that you immediately become partners with your team members, as opposed to just their “boss.” And you will make magic together once that happens.

Generative AI Is Not Going To Build Your Engineering Team For You

Charity Majors makes some really interesting points about the consequences of “replacing” junior engineers with AI:

By not hiring and training up junior engineers, we are cannibalizing our own future. We need to stop doing that. People act like writing code is the hard part of software. It is not. It never has been, it never will be. Writing code is the easiest part of software engineering, and it’s getting easier by the day. The hard parts are what you do with that code—operating it, understanding it, extending it, and governing it over its entire lifecycle.

Also:

To state the supremely obvious: giving code review feedback to a junior engineer is not like editing generated code. Your effort is worth more when it is invested into someone else’s apprenticeship. It’s an opportunity to pass on the lessons you’ve learned in your own career. Even just the act of framing your feedback to explain and convey your message forces you to think through the problem in a more rigorous way, and has a way of helping you understand the material more deeply. And adding a junior engineer to your team will immediately change team dynamics. It creates an environment where asking questions is normalized and encouraged, where teaching as well as learning is a constant.

Lots more in the full article, it’s well worth a full read.

Invested in the WFH argument? Home in on the evidence

I really don’t know how much more data we need about this. It’s almost as if RTO is not about productivity at all…

Bloom and Han reported last week in the journal Nature that, over the subsequent two years, hybrid workers showed no difference in performance grades or promotion prospects compared to office-bound colleagues; the company’s computer engineers, for example, did not differ in their coding output across the two groups.

Hybrid workers, however, reported higher job satisfaction and were less likely to quit, especially if they were non-managers, female, or had long commutes. The authors conclude that “a hybrid schedule with two days a week working from home does not damage performance”, adding the result would probably extend to other organisations.

Can Every Sales-Driven Company be Transformed to Being Product-Led?

Some solid points in this article from Jason Knight. On product-led vs. sales-led (but could also refer to engineering-led) organizations:

To be honest, my strong opinion is that if you have to worry about who’s “leading” anything, then you’ve got bigger problems to worry about than who’s leading. If your entire company is aligned around what’s important and how to get there, then anyone could “lead” you there. This is supposed to be a collaboration, not a dictatorship. If one team “leads” and others don’t agree with where they’re being led, fix that! I generally find that misalignment is one of the most important problems to solve in any struggling company.

Hard agree with that one. I don’t believe in “healthy tension” between product and engineering. Just like I shy away from the traditional hierarchical view of “bosses tell employees what to do”. Whether it’s Product and Engineering or Manager and Teammate… it’s about partnership and helping each other (and the company) grow. We have to reframe this idea that adversarial relationships somehow create better teams and products.

Why the mysterious love affair between video games and giant elevators may begin with Akira

This is a fun bit of gaming history:

The thing about games, particularly Japanese games, is that they exist in a symbiotic relationship with manga and anime. When I spoke to Uemura-san, the engineer of the NES, he said ‘Once hardware developed to the point where you could actually draw characters, designers had to figure out what to make. Subconsciously they turned to things they’d absorbed from anime and manga. We were sort of blessed in the sense that foreigners hadn’t seen the things we were basing our ideas on.’

More

  1. 1
  2. 2
  3. 3
  4. 4
  5. 5
  6. ...
  7. 200