S1E14 – The Undeserved Malignment of Bureaucracy

Show Notes

From stodgy corporate boardrooms to glacial civil-service offices, fast-moving technical organizations have long thumbed their noses at bureaucracies in their quest for speed and innovation. In this episode, Mon-Chaio and Andy unearth the forgotten power of bureaucracies and discover how they are still surprisingly relevant and impactful for software companies today.

Opening quote from “Structure and Learning in Self-Managed Teams: Why “Bureaucratic” Teams Can Be Better Learners”.

References:

•  ⁠Civilization bureaucracy quote⁠
• ⁠Structure and Learning in Self-Managed Teams: Why “Bureaucratic” Teams Can Be Better Learners⁠
• ⁠Bureaucracies Remember, Post-Bureaucratic Organizations Forget?⁠
• ⁠Path dependence⁠
• ⁠Team Scaffolds: How Mesolevel Structures Enable Role-Based Coordination in Temporary Groups⁠
• ⁠Chesterton’s Fence

Transcript

Mon-Chaio: Thanks for joining us for another episode of the TTL Podcast. Our topic today comes from a paper that Andy brought us on a very specific topic but kind of lead, at least me and I think Andy as well, toward going down a little bit of a rabbit hole which eventually led to the topic of this podcast.

So, should I bury the lede here, Andy, and just talk around it, or should I just get to the point?

Andy: We are great at burying ledes, so maybe we should just tell people at the end of the episode what they’re listening to.

Mon-Chaio: Absolutely.

Andy: No, I think we need to start this up front, otherwise we’re not gonna know what we’re talking about. So let’s make sure it’s clear what we’re talking about here.

Mon-Chaio: The article that Andy brought to us specifically talked about whether having more structure can allow a team to learn better. I believe that’s a reasonable summarization Andy?

Andy: Yeah. Yeah.

Mon-Chaio: And it got me to thinking, and I guess it got us to thinking about, huh, that’s probably true, but especially in the software world and in the Silicon Valley software world, structure or its big brother term bureaucracy is often seen as a bugaboo, a negative, the devil or something like that.

Andy: It’s what your parents did.

Mon-Chaio: it’s what your parents did, right? It’s why Microsoft lost to Google, right? It’s why IBM is a slow behemoth and nobody wants to work there, et cetera, et cetera.

And so then we sat down and we said hmm, well, is that true? Is bureaucracy something to avoid at all costs? Or, like this one paper said, and maybe some other papers that we could find, is there some benefit to bureaucracy and should we be less quick about discarding it in all cases?

Is that the question we’re trying to pose today?

Andy: I think so. What do we do with it? Bureaucracy is this thing that exists, do we just discard it? Do we embrace it fully and add bureaucracy to everything we do? Do we take a middle ground? And I think in that, the big important question is, where is the middle ground?

Where, how much do you bring in and how much do you discard? I often find these kinds of things work well, if we start out by just talking about, what is bureaucracy? What are we talking about here? I always remember the quote in, I think it shows up in Civilization. I know it’s a quote from someone, I don’t know who, but it’s “the bureaucracy is expanding at a rate to support the bureaucracy” … and something like that.

Mon-Chaio: You’re right. I think it is a quote from Civilization. I play a lot of Civ and it brings that to mind when you say it.

Andy: So since our discussion was kicked off by this paper, I’m gonna cite it and then give us the description of bureaucracy that they use, because I think it is an interesting concept that they’ve got. The paper is called Structure and Learning in Self-Managed Teams: Why “Bureaucratic” Teams Can Be Better Learners, and it’s by J. Stuart Bunderson and Peter Boumgarden from Washington University in St. Louis. And to them, bureaucracy just means highly structured. And structure for them comes from sociology and organizational theory, and in that field, structure is defined as the increased division across three dimensions: specialization, hierarchy, and formalization. Specialization is differentiation between roles and hierarchy is differentiation between supervisors and subordinates. And formalization is having explicit procedures and regulations and priorities. So that is their definition of bureaucracy. Bureaucracy is having those things defined.

But I think, Mon-Chaio, you found different definition of bureaucracy that I think was a little less, well, structured, a little bit more evocative. And I think it adds an interesting flavor to the discussion of bureaucracy.

Mon-Chaio: Yeah, it’s interesting because the definition of bureaucracy that I found doesn’t conflict with the one you found. In fact, at the end of the one I found, they talk about bureaucracy is having hierarchically defined rules and precedents that are the bedrock of continuity and consistency. To me, that means hierarchies, subordinates, managers, as well as clearly defined rules and operating procedures.

Andy: Yep. Explicit procedures. Yep.

Mon-Chaio: And let me cite my paper too. The paper that I’m referencing is Bureaucracies Remember, Post Bureaucratic Organizations Forget?, and it is by Christopher Pollitt. So that part of the definition of bureaucracy certainly fits, but they add on two other pieces, which I think are very interesting. The first one is they say that bureaucracy is a career organization with a continuous central authority. What does that mean? I think to me that means it’s an organization that’s long lived, you can build a career out of it, has a non-moving central leader or leadership committee of some sort. That’s how I read that first part. And then they also say that for bureaucracies, the written record is at the core of operations. Everything is written down, everything is documented, and those documents are the core of how that organization operates.

So those are the other two things that I would add on to the thing that you mentioned in the paper that you found.

Andy: Right. So the core of the written word and the core of stability of structure and leadership. And I think that’s very interesting because I think often when we think about those kinds of places, we think that they aren’t learning, that they’re stuck, that they can’t advance, that they can’t change.

And I think your paper had a bit of an answer to that, saying that it’s not that they can’t learn, it’s that they remember the past.

Mon-Chaio: Right. So what Pollitt would say or what he’s postulating in this paper, based on his research, is that there are some benefits of bureaucracy, and one of those benefits is this idea of remembering the past.

And he contrasts bureaucracy with what he calls post bureaucracy. And some of the ideas of post bureaucracy will seem familiar if you think about the opposite of bureaucracy: “a preference for lean, small, flat, specialized, disaggregated organizational forms”. We see that, right? Smaller teams, more fluid, et cetera. He talks about post bureaucracy attempting to link rewards to measured performance. We see a lot of that, especially in software engineer organizations. And then he makes this point, he says, “the near future becomes more important than the past.”

So what does that mean? And he goes on to say bureaucracies have a lot of what they call organizational memory. And this is the concept of the lived experiences and knowledge of the existing staff, the organization routines and standard operating procedures that have built into this culture and overall, the norms and values of the organizational culture. This is what he would call organizational memory. And he argues that post bureaucracy, because of its constant reorgs, its short-term assignments over long-term bureaucratic careers … and long-term bureaucratic careers is one thing that we poo pooh about bureaucracy, right? ” Oh, these government officials, they’re in it for 20 years. They’re just stuck there.” But he contrasts that with these short-term assignments that we see a lot. Hey, over the next three months we have this OKR to meet. Let’s get together and surge and figure out a way to impact it and move it ahead.

And then he mentions this want for this “radical, unceasing change” as a badge of honor.

Andy: Does that create PTSD for you, Mon-Chaio? The badge of honor of constant change?

Mon-Chaio: It does. To take a side detour., When I talk to people about my management philosophy or leadership philosophy, I often say I’m very dismissive of the status quo. I don’t like things to stay the same. And in one place that I worked, we were doing a bunch of this like continuous improvement stuff and somebody came to me once and in an exasperated voice, they said, we just finished this improvement project and it went well, can we just rest? Do we have to do another one right now? Can’t we just be stable for a little bit? And I said, no, we can’t. And so, yes, it does bring a little PTSD because I do think of it as a badge of honor for myself as well, being in some ways quite post bureaucratic.

Anyway, getting back on topic, Pollitt is saying that post -bureaucratic organizations have memory loss because of constant reorgs, short-term assignments, this constant change that happens, as well as the compressed timeline for high level decision makers. They feel like they have to make decisions rapidly and quickly. And so he argues that all of that means we lose sort of the stuff that he defined as organizational memory. We start to devalue the experience and knowledge of the existing staff, especially those that have been here for a long time. We start to devalue the written word. In some of the software practices, we see that too, this bugaboo that is documentation. Code is self-documenting. Documentation gets out of date, stop writing all this stuff.

Andy: Don’t write your requirements documents, don’t bother writing it down. One sentence in a Jira ticket is all you need.

Mon-Chaio: Right, and conversations are more important than the documentation and all of stuff. So that’s that first paper that I found. There’s a second paper, which is just titled Post- Bureaucracy.

Andy: They’re taking that post bureaucracy to the extreme by saying like, well, you don’t write stuff down. You don’t even need a title.

Mon-Chaio: Right. And I misread it. It’s called Post- Bureaucracy? It’s a question, by Alvesson and Thompson. They argue in their paper that oftentimes, this loss of organizational memory and the things that come along with it in post bureaucracy produces a less moral and less equitable organization.

What do we think about that? Do we believe that one, post-bureaucratic organizations do have organizational memory loss? Two, do we think that’s a good or bad thing? And maybe 2.5, is one of the bad things the fact that losing organizational memory causes you to be less ethical, and less equitable?

Andy: Okay, so is there organizational memory loss? Let’s start there.

I would say that there is, and I would also say that I think in a lot of these things, it’s almost the purpose. The purpose is to get that break from the past, with the idea that by breaking from the past you’re no longer bound by the traditions. There’s this thing called path dependency in organizational theory, which is that the place that you have gotten to is dependent on the path that you have taken to get there, and the options that are available to you at that point are also dependent on the path that you took to get there.

So I can see that trying to get rid of that institutional memory, trying to stop writing stuff, you could almost say is a reaction to noticing that path dependency is constraining and trying to find a way to get rid of those constraints. You say we don’t wanna be stuck with the burden of where we are. So we throw away that past and we throw away those documents and we break apart the hierarchy and start throwing everyone together so that there’s no specialization. And you say, there, in that maelstrom of organization, we will no longer have path dependency. I would say I don’t think it works that way because now you have the dependency on that path. And the question really becomes, is that more or less constraining to your future actions than the prior state you were in? I think that ends up being the key question to me.

And I’ll bring this back to the paper that I had about bureaucracy and bureaucratic teams learning more. And what they found there was that the bureaucratic teams were able to have more psychological safety, so they were able to bring up issues more readily than the non-bureaucratic teams. They were able to share information better, which I think is an interesting one. I think quite often when we hear bureaucracy, we think about disjointed parts. This team can’t get what this team is doing because the bureaucracy stops them from communicating. And it may be true at a large organizational level. This paper was studying individual teams and how much structure they had, but they found, at least at this smaller level, that having some structure helped them communicate and helped them share that information.

I don’t think we can say that the post bureaucracy is, that they’re trying to learn more. I don’t think they’re trying to do that. I think it’s what they’re trying to do is they’re trying to break themselves from the necessary constraints that the past puts on them.

Mon-Chaio: I do think that you’re right. I would also note that it’s not just chopping ourselves off from the past. I see it as a devaluing of the longevity of information.

Andy: Yes. I think that is an explanation to why it’s seen as legitimate. To chop the tail off so that they can run free. It’s devaluing the tail.

Mon-Chaio: When we talk about bureaucracy, we often see that tail, right? Oh, this is how the company behaved 10 years ago. This is the things that they found 10 years ago, it’s not relevant now, but the tail is shorter than we think.

You see a lot of these software organizations that will tell you the tail is three months, maybe six months. The devaluing of documentation shows you that. By the time you write the documentation, it’s out of date, so why write it? Encompass it into executable code because code changes all the time, and so your docs will change all the time. So I think there’s even a devalue in information that most would consider short term, not that long-term bureaucratic memory or information from years into years ago.

Andy: Okay. So now moving into your second question, which is, is the loss of that history or the loss of that information bad, and is that then connected to a less ethical or less moral approach to work? I think that’s a tough one. To me, that starts becoming very context dependent about is it bad, is it not?

But I think in general, done blindly, yes, it is bad. If your entire motivation turns into, cut off that information, move into post bureaucracy because we don’t wanna be bound by it anymore, that starts running afoul of understanding why certain restrictions might be in place.

So for instance you might be working in a software team where they say in order to connect to the database, use this library that wraps the JDBC calls and it removes certain things that we don’t want to do. And you might say, oh, but it doesn’t let me do this thing that I want to do. I’m just gonna throw that away and I’m going to write my own thing. I’m getting very down into the details of code here. I’m gonna throw that away, but maybe you’re missing that that library was put in place because every other system depends on you having those behaviors. And if you don’t connect to the database and deal with it in that way, when a failure scenario comes along, your application will make it worse. And that was something learned through multiple failures, multiple small tweaks over time, over like 10 year history. And you’ve just thrown it away without understanding it, and you’ve made your organization worse.

Now, at a moral level, at an ethical level, the exact same thing will happen for inequalities. So, you say, oh, this way that we do interviewing, I don’t like it. I wanna ask my question. I don’t wanna ask the question that the company has come up with about how we’d assess the quality of a coder or whether or not they work well with us, I just want to ask mine. Because these HR people don’t know what they’re doing, and you have no knowledge about how to make sure you remove all sorts of biases from your approach, and so you are inadvertently, by saying just throw away the past, getting rid of that bureaucracy of how to do things, the separation of roles, and the written procedures. You’re throwing that away and now you are introducing the inequalities that those were attempting to get rid of.

Now we can debate whether or not they did or not, but you might not even be aware that yours are going against everything that is known about how to get rid of those inequalities. So I think doing it blindly becomes unethical.

What do you think Mon-Chaio?

Mon-Chaio: I think it’s almost impossible to not do it blindly.

Andy: Hmm.

Mon-Chaio: But I think the intent must be that we want to try to remember some of the past. Unless you’re explicitly stating that in some small cases you really do want to divorce yourself from the past.

But I think our biases are always moving us forward, right? Especially if we were brought up in the Silicon Valley way, and I think society as a whole moves this way. Our goal now is not to research everything that brought us to a certain point. We take that point and then we put our own spin on it immediately. I’ll give you an example in the US: people no longer know why you shouldn’t step into a crosswalk when the crosswalk light is blinking. There is actually a rule and a formalization for why that is. The answer is that we want to give cars who are right turning space before the next light happens to turn, and so pedestrians shouldn’t be in the crosswalk, or let’s not cross late, right? So we can give a window for cars to turn right or to finish. Now, one can debate about whether that’s valid or invalid, whether we should give pedestrians more space or less space, but deciding that you’re gonna break or change that rule without understanding where it came from is problematic.

Andy: Yes, I agree with that. I was gonna bring into this the idea of Chesterton’s Fence. Do you know Chesterton’s Fence?

Mon-Chaio: I don’t. Tell me.

Andy: So the idea of Chesterton’s Fence is that change should not be made until the reasoning behind the current state of affairs is understood. And the idea, it’s apparently a bit of a story, that there’s this fence and people come up to it and they’re like, this fence shouldn’t be here. And they wanna remove it, they want to change it, but no one knows why it’s there, but they still wanna get rid of it. And it’s just this idea, before you remove the fence, You should know what the fence was attempting to achieve. Now, it may be that it never achieved that, it may be that it doesn’t need to achieve that anymore. It may be that it’s something that we don’t ever want to achieve anymore, but you should still understand what it was trying to achieve before you just rip out the fence.

Mon-Chaio: And I think that if you follow the intent of Chesterton’s Fence, I think that’s great. But I think in the modern world, the history becomes much more convoluted.

It’s no longer, oh, that fence was put there to keep cows out or to keep wolves out or something like that. It’s, it was put there to keep wolves out, but then there’s this ecological thing around a river flowing through, and so then it had to be moved over here, but then there was erosion that happened here, and there was a property line that was redrawn there, and …

So I think in the modern world, the variables become multiplied from what it used to be. I think it’s really difficult to live your credo by Chesterton’s Fence fully. And so I think that’s why the intent is so important. You want to intend to live by that, but you can’t say, look, I can’t make any change until I completely understand that. Because that might be a year or two of research for everything that you intend to change.

But I do think there’s problems with it. You were talking about the low level details of software.

Look at how cyclical software is, right? Software people often say software has no memory. Do you remember the compiled worlds and strong typing of C++, and then the move into languages like perl or python or php where this dynamic typing thing was all the rage, and now we’re back into go and static typing.

We could talk about things like how dependency management is think about JavaScript dependency management and the number of dependency that you have for every single project, right? Remember …

Andy: Let’s not talk about that. Let’s not talk about JavaScript ecosystem.

Mon-Chaio: So that brings up your PTSD, although your stress was much more recent. So I think we can get into these patterns where we forget about the past, and I think it does cause us some detriment, even though it does provide some benefit as well.

Diving back into the equalities thing, I do think that if you don’t follow Chesterton’s Fence, you can get into problems with equality. Just look at how many Silicon Valley companies get into trouble with regulations and fines and running afoul of the law versus long established companies, right? A lot of that I think, has to do with this concept, the post-bureaucratic thought, listen, the old ways aren’t working. I’m not willing to completely understand why they’re not working. I’m simply gonna put out a product in a new way, and by letting it run, I will learn. And that’s my learning, instead of digging down into the history of why things were.

Andy: So it can be problematic. I think we’re agreeing, people need to put some effort into understanding the intent behind the structure before simply throwing it away. Recognize that you aren’t going to necessarily understand all of it. So there has to be a bit of courage going into it, but not bravado, not the bravado of ” I know, and just throw it away”. We want courage and humility.

Where does this take us? What kind of tactics, what kind of things can we recommend to people that they do with this newfound knowledge about bureaucracies are not evil, structure can set you free, can lead to better learning. What kind of rules of thumb could we possibly give people to use here?

Mon-Chaio: I think one is to use structure or bureaucracy where it can shine. We’ve talked a lot on this podcast about the concept of explore, expand, extract, or the town planner model of thinking about work, software product, software product organizations.

I think when you’re in the explore phase, you’re looking to learn by exploring. You don’t know what’s out there, and the past may constrain you in those cases where you have to move fast, where the shape of your learning and the shape of your product can change on a whim. So I think in the explore phases, and possibly in some of the expand phases, I think you can look to post-bureaucratic techniques to move you along.

Andy: Yeah.

Mon-Chaio: I think however, once you hit that extract phase, and I think the paper that you were talking about would point to this as well, where it mentions they study teams that were doing, stable tasks, right?

Andy: Mm-hmm.

Mon-Chaio: They also called it repetitive tasks. I want to take repetitive out of it and just focus on the stability part. Oftentimes in the extract phases you are doing stable tasks. Not to say that there’s no innovation in that phase from those tasks, but the innovation is not around, I’m gonna right turn around a corner, I’m gonna u-turn immediately in the next week or in the next month.

And so I think in the extract phases, you should put bureaucracy into place and you should put more structure into place because as some of these papers have mentioned, it gives you more organizational memory. It possibly gives you better ethical behavior. It gives you better psychological safety, and it gives you higher information sharing because of that. And so I think my tactic that I would recommend is, think about explore, expand, extract, figure out which teams and which parts of your organization are in which phase, and don’t be afraid and actually enthusiastically apply bureaucratic techniques to those phases where it can actually benefit.

Andy: I’m gonna emphasize that last point again, that different parts of your organization, and different teams, possibly even interacting with each other, might be at different points in that sequence at the same time. And so you don’t want to apply your bureaucracy evenly. It’s not like jam on a peanut butter and jelly sandwich. You don’t want it all spread evenly around everything. You want some parts that are a bit different, depending on what the different parts need to do. And don’t be afraid to change that a little bit as they change what they’re trying to do. But also Chesterton’s Fence. Try to understand why some of them are in those particular ways of working.

And now I’m gonna add onto that, at that explore phase, that’s not to say you get rid of bureaucracy. Because one of the things that we found in our reading again and again, was that the bureaucracy helps.

This is I think the really hard thing is a bit of bureaucracy actually helps teams. So we found it in a paper that was talking about what was called team scaffolds, and there it was for teams that are in an ER or an emergency department. And they’re changing constantly. They’re constantly changing who’s doing what and what the members are and what they’re working on. And you might say, and they thought, you don’t put structure on it because it’s too fluid. And what they found was a little bit of structure, what they called a team scaffold really helped. They got actually 40% better at getting their patients through. And the exact same thing can happen in software.

A little bit of structure can get you much more effective. And their guidelines, their attributes that they were looking for in these scaffolds, was that it needs to have boundedness. We’re getting back to the BART and the boundaries. If you haven’t listened to our episode on BART it was our very first episode we published. Back to boundaries. So it’s boundedness, which is that you’re explicitly clear who’s part of the team at any given time. It’s about being clear about the role set, which is that it is a group of interdependent, de-individualized — it’s not people, it’s roles — with the complement of skills needed to produce the joint deliverables. So at some point, Mon-Chaio, we might talk about this idea of staff liquidity and keeping track of what skills are needed for a job and what skills are available. But that is a clear part of the small amount of bureaucracy you’re going to want for a really effective but loose, almost pure post-bureaucratic team.

Mon-Chaio: Mm-hmm.

Andy: And they wanted collective responsibility, that those on the team are responsible together as a group for their product or outcome. And they found that if your team scaffold creates those attributes, and you just need enough to create that, then they get much better results.

And I think the other aspect that they didn’t find, but another paper did, is around hierarchy. Now, hierarchy I think is another one of those things that we are often . Very reticent to talk about and bring in. And it seems like the ultra-bureaucratic thing of like saying, who maybe has some authority over someone else. And I think there’s two things.

One is that the bureaucracy paper that started us off on this whole thing, they addressed hierarchy and they said that hierarchy does not automatically compromise psychological safety in teams, but rather it depends on how hierarchical status is leveraged and specifically on whether those with higher status adopt a participative approach.

And so the hierarchy clarity, if you can get clarity on what that hierarchy is, and you can train the people to take that approach. What they found was becoming clear in everyone’s mind what that hierarchy is that they’re dealing with, it also reduces conflict. It increases performance. So it’s another thing to reinforce. You don’t need to set down the strictures of everyone knows exactly what their role is and they cannot deviate from it.

But if they have those ground rules of what is the team, how are they expected to work together, what is their goal, then that is the minimal amount of bureaucracy you need for those really effective teams. So don’t throw it all away. Don’t just be like, hey, put this 10 people in a room and leave them to it. No. Help them find that minimal structure to actually work together.

Mon-Chaio: I like it. And in the initial paper that set us down the road for this topic and this episode, they mentioned that often the challenge of bureaucracy is these top-down bureaucratic processes and hierarchies that reduce psychological safety and autonomy via their structure. And so I would say, think about team boundaries, think about hierarchies. Think about, to what you were saying, Andy, letting everyone know who’s in charge. That is so important. Not everybody could be in charge, who is the leader, and bureaucracy would say, yeah, you have a leader, you have a clear leader. Now they would also say, have a long-standing leader. I don’t think we need to go that far. Especially in, maybe explorer groups, but make it clear who’s a leader.

So put structure in place, hierarchies in place, understandings and processes in place, that don’t compromise psychological safety and autonomy. And we’ve talked a little bit about AAA teams, right? I think AAA teams are a lot like that. You have autonomy, everyone’s together for a clear goal. So think about bureaucracy, I think, in that aspect as well. Think about those bureaucratic practices that you put in place and whether they are affecting psychological safety and autonomy.

Andy: Is there anything else? Did we skip anything?

Mon-Chaio: Yeah. Andy, I think I’m gonna be clear that I am above you in this hierarchy, and so my decision right now is that we have done everything that we need to talk about in this episode.

Does that help your psychological safety?

Andy: It does. I’m quite happy to cede that authority to you. I don’t like closing us out, but I am really interested in what people think of this. Are we off on the wrong track on thinking about bureaucracy as the sliding scale through the organization? Do you disagree with us about the impact that the post bureaucratic world and leaving behind all of that history has on things? Is it moral? Is it ethical? Is it immoral? Is it unethical? We’d really like to hear, so get in touch.

Mon-Chaio: All right. Just another quick reminder, if you enjoy what we’ve talked about, definitely let us know what other topics you would like and rate us on your favorite podcast platforms. We really appreciate your support. This podcast can only grow and has only grown due to your support, so thank you very much.

And until next time, everyone, be kind and stay curious.