Show Notes
In this episode of Tactics for Tech Leadership, Andy and Mon-Chaio dive into the discipline of organizational diagnosis through the lens of the paper ‘Organizational Diagnosis: an Evidence-Based Approach’ by McFillen, Balzer, Varney, and O’Neil. They discuss the need for a rigorous approach to diagnosis in organizational design and change. The focus is on understanding the four spheres of knowledge: standards, solutions, symptoms, and systems. These spheres provide a structured approach to making evidence-based diagnoses. The hosts emphasize the importance of coherence in systems and how teams can implement this structured thinking into their practices to ensure organizational health. The conversation provides insights and actionable advice for technical leaders aiming to improve their diagnostic practices and organizational change efforts.
References
- Organizational Diagnosis: An Evidence-based Approach – https://www.researchgate.net/publication/239796277_Organizational_Diagnosis_An_Evidence-based_Approach
- The history of the theory of the circulation of the blood – https://journals.sagepub.com/doi/full/10.3233/CH-168031
Transcript
S3E12 – Organisational Diagnosis
Andy: [00:00:00] Welcome back to the Tactics for Tech Leadership. What are we gonna talk about today, mon-Chaio? Is it more about organizations?
Mon-Chaio: I think it is. That is our theme for season three, isn’t it?
Andy: It is. And this time, what happened was we finished up our last episode and Mon-Chaio and I were sitting here and thinking about where do we go next? We’ve run out of the things that we know off the top of our heads, or were immediately evident from the things that we had been reading. And we thought, you know what?
What we haven’t done yet is in the discipline of organizational design. How [00:01:00] do they think about diagnosis? So we did a very quick little Google search as you do, and we came up with what looks like a very interesting article. And so, today we are going to be talking about that article, which is drum roll, please.
Organizational Diagnosis an Evidence-Based Approach by McFillen, Balzer, Varney, and O’Neil. All from Bowling Green State University.
It pulled us in. The authors did a really good job of writing a title that got our attention.
Mon-Chaio: And so that’s great for the title, but like, can we give readers a sense of why this paper was written, um, and what they are talking about, why they felt like this was important to communicate.
Andy: Yeah, so the overall idea in the paper, and I’ll say [00:02:00] this is not a research paper, this is a paper setting out their thoughts and a, and a proposal for where as a discipline, um, they need to go. It proposes that the organizational design and change OD&C discipline is lacking a rigorous approach to well diagnosis. And they look at as we did, they actually look at medicine, uh, they look at differential diagnosis and they, they give an explanation of where did it come from, what, uh, critical pieces come together to make it work, and what was necessary to allow those pieces to exist. They also talk a little bit about the kind of what they call engineering diagnosis approach. Which is a very simplified thing. Did you, did you catch [00:03:00] that? It went by very quickly.
Mon-Chaio: I did. I, um. I really liked it. I was like, Ooh, great, the engineering side. And then they said, well, but you know, medicine’s better for organizations.
Andy: Yeah.
Mon-Chaio: and you’re right, they took a very simplified approach. They, I think when they were saying that medicine was a better approach, they compared the two and their, their conclusion was that engineering has much more defined sets of symptoms and rules.
And that diagnosis is easier because of that. Whereas in medicine and in organizations, there’s all these different spheres of knowledge that have to come together that are much broader. And it’s rare that one practitioner can know everything about all four spheres of knowledge as they talk about, which I’m not sure I agree with, but.
Andy: I, I, I can agree with them that, uh, there’s a lot more unknown [00:04:00] unknowns. A lot more unknown interactions
Mon-Chaio: Yes, I can agree with that. Absolutely. Right.
Andy: In most engineering systems. We can eventually pull the thing apart and understand exactly what can cause, what
Mon-Chaio: Mm-hmm. It’s a little more, yeah, it’s a little more predict, uh, that’s not the right word. Whatever it is. Um.
Andy: deterministic.
Mon-Chaio: a little more deterministic.
Andy: Yeah, so they take us through these steps that the medical profession goes through, and actually they took it through a much more fine grained description than we ever went into. So, I’ll, I’ll call out a little bit of what they consider the diagnostic approach in medicine and where differential diagnosis fits into it, because it’s not the full
Mon-Chaio: Right.
Andy: But what they say is that it goes through about a, what is this nine step process
Mon-Chaio: Mm-hmm.
Andy: where you have data acquisition, getting the symptoms and history, physical examination, lab tests, that kind of thing. The [00:05:00] second step is accurate problem representation. That’s where they’re trying to give names, give labels to some of these things that they’re seeing.
Then they get to differential diagnosis, which is hypothesized possible causes, saying like, well, it could be this or it could be that. And then they prioritize different diagnoses of that differential diagnosis. Then they perform tests of those hypotheses that they’ve come up with. Then they review and reprioritize the differential diagnosis based on that new data.
Then they revise the differential diagnosis, then they test the revised differential diagnosis and they confirm a final diagnosis. Now that makes it all sound like it’s only you go through this twice and then you’re done. I think. I think that basically they could say that steps, uh, 1, 2, 3, and four are done in a [00:06:00] loop
Mon-Chaio: And in in the paper they have a little chart and it definitely goes from diagnostic tests back to data collection. The iterative review and revision before the final diagnosis.
Andy: Yeah. Yeah. You just kind of iterate through that again and again until you get to a point where you say, I’m pretty sure I found what I have here. What’s, what’s going on here. But the thing that I think was interesting and for us to really talk about is how they then take this and kind of give us a metamodel.
They give us this metamodel of what does a diagnostic function need to have? And that was, I think that was the thing that really caught our eye. They talk about what they call spheres of knowledge. So let, let’s start digging into that Mon-Chaio.
What, what are your just top level thoughts on this metamodel of these spheres of knowledge?
Mon-Chaio: Well, I think first of all, we need to talk about what the spheres of knowledge are. Yes. Um. So for them, the [00:07:00] spheres of knowledge in a diagnostic system are standards. So this is what is the, what is supposed to be, I guess in the medical world, they would say standards are things like the body temperature needs to be 98.6 degrees.
That is a standard. And of course, standards can evolve, right? As people age, body temperature changes. So you have different ways of modifying the standards, but they’re all known standards.
Andy: When I mentioned this to my wife, she said, I think she used it a a better word than standards. She said, well, you need to know what healthy is.
Mon-Chaio: mm-hmm. Right. And they even might’ve mentioned that in the paper. That sounds like something that I remember reading. The definition of healthy, I think is, is great, right? That’s the concept of standards. So that’s one sphere. Standards. The next sphere is solutions. And solutions is how do you affect change, right? Like if you have a problem, what is the solution? And again, in the [00:08:00] medical field, because it is quite, uh, it’s been around for a lot, it’s been around for quite some time.
There are a lot of standardized solutions. You can think about, oh, if your fever is X and you present these symptoms Y, then maybe you take ibuprofen for three days and then you call your doctor if symptoms haven’t improved after six, something like that. That’s a solution, right?
Andy: If you have this kind of bacterial infection, you take penicillin. If you’re allergic to penicillin, you take this.
Mon-Chaio: The third sphere that they talk about is symptoms. I think that’s pretty self-explanatory. How does the patient present? Or in an organizational case, which is a little bit vaguer, but how does the organization present? What are the symptoms? And this gets into the data collection part of it.
How do you collect data about the symptoms? And the last part is sys.
Andy: I, I wanna let, let’s go into symptoms just a second. Because there’s a thing I wanna call out about it, which is that symptoms can’t exist, [00:09:00] at least as far as I was understanding here. Symptoms can’t exist without that definition of standard.
Mon-Chaio: I don’t know that that’s true. I think even without a definition of standards, symptoms can exist. And when we, in my opinion and when we go deeper into organizational diagnostics, I would say that that’s one of the problems is. Few standards, but lots of data collection around symptoms. So I do think so as for the body
Andy: I’ll hold on. I’ll hold on to my slight disbelief of that for a moment.
Mon-Chaio: Okay. All right. So let’s touch on the fourth one. So the fourth one is systems and systems would be the understanding of the way that I think about it, the model that you’re trying to adhere to. So in the medical field systems might be human anatomy and physiology, for example. That’s a system. And you understand how the components of it are designed, right?
Like joints fit into sockets, [00:10:00] um, and muscles do a particular thing. That’s a system. And so you know, in the medical field, obviously there’s many symptoms. Um, and they mentioned that early human symptoms were inaccurate, right? Like there was a belief that the heart was the locus of human emotion, right?
That was a, uh, was a system. it is now no longer a system because we have evolved our systems knowledge. But that’s.
Andy: Did you know? Did you know that there used to be a system, this was a, I think what the ancient Romans believed, which was that your blood started at the center of your body, migrated outward, and then magically left your body. I don’t understand how it, it made no sense to me, but that they had this idea that the blood was flowing outward and would just kind of dissipate out of your body.
Mon-Chaio: I like that. Interesting. Well that is, yeah, that is system that is no longer believed. Yes.[00:11:00]
Andy: Yeah. Yeah.
Mon-Chaio: So those are the four spheres systems, standards, solutions, symptoms. And when you bring ’em all together, that allows you to use evidence to make diagnoses. So that’s the spheres of knowledge and their use, uh, and their connection to diagnosis.
Andy: Yeah, and if any one of them is missing, I think what they would say is you can’t do evidence-based diagnosis
Mon-Chaio: right.
Andy: So if you are, if you’re missing what those standards are, what healthy is, well, you have no evidence on which to go for what solution you should be using as an example because, well, any one of them could be correct because there’s no particular direction you should be trying to take it. And if you don’t understand the system, then those symptoms. It can mean anything to you because they’re not telling you what’s happening within a causal system. And if you don’t have [00:12:00] any solutions, well, you have no way of affecting change on that system to change those symptoms into, towards the standards that you know.
So if any of these are missing, what you end up with is a, a thing where your diagnosis can’t happen and it can’t be based on any evidence.
Mon-Chaio: Right, and I would say that. There’s probably little disagreement that we need all four of these spheres of knowledge in order to make diagnoses. I think what the paper would purport is that there are good to not great research in each of these spheres and, whereby there is good research, we should use it. And whereby there is not good research, we should do more research.
Andy: Yeah, and one of the things they pointed out was that it’s hard sometimes to even find what is the good research, what is the current, like full understanding of what’s the system of an organization or what’s the current [00:13:00] understanding of what are the effective solutions that you could try applying. They said they did point out that there are several.
Places where those have started to come together. Like some authors have done research where they’ve pulled together. I was looking at one of their references, which was, uh, taking the Measure of work, which is a guide to validated scales for organizational research and diagnosis sounds very useful as a way of measuring to find those symptoms.
Mon-Chaio: I agree, but I think this is, like you were saying, when we skimmed the paper, this was, this kind of jumped out at us as something that interested us and made us think, wow, this paper is probably worth reading. It’s probably worth discussing. Right.
Andy: Yeah. So, so now we get to the question of was it, now that we’ve read it and talking about it, was it useful? Was, was it something that we can. We can use and we can and guide our own [00:14:00] practice with this.
Mon-Chaio: I think for me it was, I had not thought about sort of the four spheres of knowledge before as part of diagnosis. But of course anybody that does diagnosis kind of stumbles upon it, even if they don’t have the words to talk about it. Reading the paper, the first thing that jumped out to me is using the system sphere of knowledge as a way to explain why many organizational change efforts fail.
Andy: Can you say a bit more about that?
Mon-Chaio: My feeling is that most people doing organizational change or diagnosing organizational change, especially if they don’t come from the quote unquote consulting field. Right, everybody kind of does it a day-to-day job or, um, as they advance through leadership. I would say most of those people don’t think in terms of systems and have [00:15:00] very little knowledge about organizational systems. And so when you don’t have a system, you don’t kind of know where you’re aiming for, right? Like we can talk about standards as an aiming point too, but. The systems are a bigger aiming point because you don’t know how various pieces fit in. If I change this piece, how does it affect that piece, right? Like if I introduce work from home four days a week, how does that affect culture?
Right? Is there a system that tells me the difference between face-to-face communication and its impact on culture? Or if I change my performance review process to do 360 reviews, how does that change X, right? People don’t know these systems and they, so they make them up and so they just say, well, mm-hmm.
Andy: I think they make them up, I think. I think we end up doing a lot of [00:16:00] ad hoc experimentation, so we kind of do this thing and say, well, I want this to happen. And maybe you get a little bit of that, but not much. And then you notice that all these other things started happening, but you don’t know. That was that related to what I just did here?
Like maybe you start saying like, every Friday we’re gonna have a presentation from members of the executive team talking about what our goals are, what our OKRs are, and you start noticing that. Well, yeah, people are staying a little bit more focused. We’re hearing it more in our meetings that are, they’re talking about those goals, but we’re also getting this thing of people are kind of checked out on Fridays. Is that connected? Is, is there something going on here? We don’t know. We don’t, we don’t have a system that would tell us why things might happen when, when we, when we do something
Mon-Chaio: That’s right, and I [00:17:00] can’t remember whether we touched on this season or not, but I think the trouble with not having a system also is that you are not all moving kind of in the same direction. You don’t know where good looks like, and we’re starting to get into standards
Andy: to me that’s, that’s standards. It’s, and I think that’s, that’s an interesting one for organizations is are there universal standards? Like, I think there’s also a question for, for systems. Are there universal systems? And I think there are probably some, but I don’t know if they exist for across all industries.
Mon-Chaio: I.
Andy: It’s like, is the system of a a steel foundry, is it the same like organizational system dynamics as the, uh, as an AI research company?
Mon-Chaio: Right, and I think we can pretty confidently, at least for me, say no. I think one of the [00:18:00] biggest challenges in software engineering that persists to this day is taking a system where you’re building houses and standards where you’re building houses and from the construction industry
Andy: Mm-hmm.
Mon-Chaio: it on top of software and saying, well, you know, uh, there’s an architect that architected the house.
So we have to have an architect that architects the software. And what does good architecture look like in a house? Well, there’s all these change documents for anytime you change something. So we have to have change management documents that flowed from a different system that was kind of brought in, um, I don’t think represents the industry well.
Andy: Do you think that the amount of cargo culting that we find in software development organizations is because the standards that do exist are not evidence-based.
Mon-Chaio: I do, I do. They’re not evidence-based or [00:19:00] systems-based, I would say.
Yeah.
And so I do think that when people look at symptoms like, oh, my code coverage number is under 50%. And then they look at standards which said, Hey, look, if you look at Microsoft, they make their people do a 75% code coverage number, aren’t they a successful company?
Mon-Chaio: Then they jump right to solutions. Right. Whereas like one, is that even a real standard, like it is a one company.
Andy: Is, is there any relationship or are we just taking that the equivalent of, well, this person doesn’t have cancer and they have a full head of hair, therefore I need hair transplants to, to not get cancer.
Mon-Chaio: Right. Absolutely. I think, I think it very much is that for most organizations
Andy: does that mean I need to stop? Like propagating, like extreme programming and TDD?[00:20:00]
Mon-Chaio: well, that’s, I think, I mean the, um. That’s a good example, right? I think a lot of the practices you can, you can say are standards, um, this idea of, oh, you always pair program or you write a test before you write code. That’s kind of a standard in the process sphere, but it all comes from systems, right?
The question is, why do you do that? And we. You and I think always do try to do a good job of leading back to, okay. The reason why is because fast feedback loops, right? That’s part of the system. Fast feedback loops allow us to x, y, or Z. The system tells us that that’s true.
Andy: I think, I think that there’s actually a thing here, which this has given us a language to talk about it this way, which is where I think that this is really useful. I, I like these models, not because that they’re necessarily giving us an answer, but because they give us a structure which we can use to explore the problem.
So if
Mon-Chaio: Mm-hmm.
Andy: we take this and we say, look, there’s, there’s an issue that we have in software [00:21:00] development or the lack of standards by this term. We have plenty of standards. There’s all the ISO standards and, uh, RFI, uh, RFPs, the internet standards, whatever they
Mon-Chaio: Right.
Andy: RFCs, RFCs. And when we’re thinking about our organizations though, we don’t really, we have these ideas of what a well-functioning, healthy organization looks like. So like XP, like you were just saying, there’s all these practices and you do these things. And in a way it’s because you do those things because you want to be running a particular system. You want particular sets of interactions. Now I think, I think I disagree with you. I think there is a universal system. It’s the universal, how do humans behave? So it would be getting into psych psychology and sociology and behavioral studies. And I think [00:22:00] that’s probably the system, but what we see is the system that gets layered on top of that.
That’s much easier for us to see. We think about like, what is the system that has been written down in terms of policies and procedures, and we decide that in order to be healthy, you have to set a particular system, but the reality is all sorts of systems will get you to that healthy state. And what we completely lack is clarity on what that healthy state looks like.
I think the DevOps research gave us a glimpse of it. But we don’t have many, we don’t have many standards of what they look like. And what I see is that then people take those standards of this is what a healthy organization looks like. And one of it’s like you deploy five times a day. And what I see is people then set up systems where they just deploy five times a day. They just say, well, we will [00:23:00] do that thing that’s healthy. And what they end up with is it’s, it’s like if someone had a problem where they couldn’t maintain their body temperature at 98 degrees. So just put them in a bathtub with some hot water and bring them up to temperature. There! They’re healthy now.
Mon-Chaio: I agree with everything except the part where you talk about people inventing systems. ’cause that’s not really how I think about systems. I think about systems as a model. To understand the interactions between different parts of the organization and how that creates results. Where we can make hypotheses and understand the cause and effect of different areas because the system tells us this. I do agree that the top level system is human psychology and human interaction. Do you recall, Andy? We once had a brief discussion [00:24:00] about. How that might be a little bit dangerous to assume that that’s unchangeable.
Mm.
I don’t know which side I lean on. I think I lean more on the look we’ve been through so many years of evolution.
Mon-Chaio: It’s probably mostly unchangeable except around the edges. But I can also subscribe to this idea that, yeah, maybe it’s more malleable than we think. And just saying, for example, you can’t build trust in under three seconds maybe isn’t real. Like if we just tried, maybe human brain is more malleable, whatever the case may be.
But I do agree, like there’s a system in place around human psychology, but I think underneath that, most of the systems are missing. And so then it just becomes, in my mind, a standards based approach. You have these standards, you have to deploy five times a day, so you go deploy five times a day.
Andy: Mm-hmm.
Mon-Chaio: But you don’t understand what the interactions are around those deploys and what it might trigger, and you don’t have a system to think about it.
And so when that creates more symptoms, you [00:25:00] just simply look back to your standards pool to find another standard that you then action a solution on.
Andy: yeah, yeah. So on that, uh, you deploy five times a day. I’ve seen teams where they deploy five times a day, but the software never works. And the thing that you’ve put in place has made a bad outcome, you are not closer to your standard because the rest of your system can’t support it.
Mon-Chaio: Yeah, I have a, uh, I have a really funny example along those same lines. I once had a team that told me they were at a hundred percent test coverage, and I may have told this story before. One day I was pair programming with this engineer and they were struggling around this new logic that had to go in and, uh, as most engineers do, which Andy you and I both dislike, he looks at pieces of code and he reads it and he thinks, and he takes out his notepad and starts drawing little diagrams and he goes to this other part of the code and it’s [00:26:00] 25 minutes. He has not typed a single line of code, right? He’s just like reading and like diagramming or whatever. And I was
Andy: I’m, I can go along with that for, for a while.
Mon-Chaio: I was like, wait a minute, you didn’t, you tell me you have like a hundred percent test coverage, so why don’t you just like write the code you think should exist and then run the tests.
It will tell if you broke anything. He is like, oh, no, no, no, no, no. I can’t trust the test like that.
Andy: I can’t trust. Oh, yeah. I can’t trust test to actually tell me if it’s
Mon-Chaio: Right. So you met your standard, but like, what does that even mean?
Andy: I think it’s, it’s the, I think in the terminology that they’re using here, it’s not that they met their standard, it’s what you were saying. It’s, they met their system. They were executing the system that they had. And the system that they had was that they write these particular tests that gives them a hundred percent on the score that they’re on the, on the measurement that they’re using.
And, [00:27:00] but the standard, the, the standard of a healthy team, I would say is that their tests fail when their sy, when their code doesn’t work.
Mon-Chaio: Yeah, I don’t know that I agree necessarily with that. I think that, I mean, I think just like, you know, body temperature, 98.6 degrees or whatever it is . A standard could be 75% test coverage. Right? That is a measurement that we can do to say that is the standard of a healthy team. Um, now of course there’s higher level standards that you can, you know, there’s hierarchies
Andy: you can. You can have other standards on there to say that. Well, yeah, but we all know that 75% test coverage doesn’t mean you catch
Okay, so your tests fail when there’s a problem.
Mon-Chaio: I think it’s not that they’re executing their system, it’s that they have a lack of a system. They don’t have anything to go back to to say, well, I have 75% plus test coverage, but I still can’t trust the test, what does that mean? They don’t have any model to [00:28:00] think about that interaction about why that may or may not be a good thing.
Andy: And I think there, what you’re getting at, and I’ll, I’ll, I’ll, I’ll concede to you. Yes. Okay. Most of the systems that they’re talking about is more of these organizational components and how they interact and, and that there’s always a system, even if you don’t know what it is.
Mon-Chaio: Okay. Okay.
Andy: And, and I would say that that’s one of the things about this as the paper, what they’re saying is you actually need to put in the effort to find out what that system is.
Mon-Chaio: Yeah. Yep.
Andy: Because if you don’t have that system, if you don’t have a, a, a reasonable description of what that system is that people believe that they’re operating in, well, you’re, you’re not gonna be able to do a reasonable solution based on symptoms that are cut, connected to important components of that, of that system.
Mon-Chaio: I, no, I agree. And we kind of touched on this earlier where you said like even people that don’t have names for them [00:29:00] will invent them because. They’re kind of necessary in order to exist. They can’t be completely absent for you to actually be able to do diagnosis. But when you invent them, like how structured they are and how good they are, I think is generally
Andy: they coherent?
Mon-Chaio: Are they coherent is a big one. Or you jumping from system to system and standard to standard depending on the day? Um, I, I think the one other problem with organizations is. Some of them aren’t even standards based in terms of executions. Forget the systems part of it. They’re not even standards based. I feel like a lot of ’em are just outcome based.
It’s like saying, Hey, this person who weighs 450 pounds was able to run a sub four hour marathon. Therefore they’re healthy because they finished the marathon.
Andy: Yeah.
Mon-Chaio: Right. Like, as long as I ship products and my cust and I make money, the system or the the [00:30:00] organization must be healthy. There’s not even a standard for it.
Andy: Well, you could say that, well, it, it, it would be similar to saying that you’re healthy because you have a, an appropriate body temperature. Like you can say that is a standard, like running a four hour marathon. That’s pretty good. But it’s probably that there’s aspects of the standard, and this is what they’re calling out for the research, is we need more descriptions of what healthy is an organization. And, and I think, I think in software development, we are getting a little bit closer, slowly.
I think once again, the DevOps report has helped, I think the, what is it? The space. Uh, research has probably helped a little bit as well. What I think we don’t have is standards of more interaction based stuff. ’cause I think both of those are fairly, one is very [00:31:00] much the, like, deployment and operation of your software.
And the other one is very, I think I, I, I, for, it’s gone out of my head for space. I think fairly individual developer centric. I think what we don’t have a great idea of what the standard is, is, well, what is, what is like that team look like? Or what is it that that department looks like when, when they’re getting, when they’re healthy, when they’re doing well?
Mon-Chaio: It’s my current sprint at my job, uh, is around, uh, KPIs for the technology organization. And so I actually tasked my chief of staff with reading about space and Dora, and then presenting what she thinks is gonna be a reasonable KPI framework for the org.
Andy: Yeah, you’re, you’re trying to come up with what, what does healthy look, what, what does healthy look like and what are the measures you can do to ensure that you, you, you can measure that you’re healthy by, get some evidence that you’re healthy.
Mon-Chaio: Absolutely. That’s [00:32:00] exactly right.
Andy: Nice.
Mon-Chaio: And so getting back to the article though, I do think that, you are right in that we we’re starting to get a better sense of what healthy looks like. But I don’t know that, Hmm, how do I say this? But I don’t know that without agreement on a system that can really move the needle. Because I do think that, and, and this is gonna be a challenge actually in my current organization too, I believe. I think that we can come up with KPIs and what healthy looks like.
How you get there also matters. And how you get there, I think is very systems driven. An example I’ve been thinking about recently is, well, you know, part of space is collaboration, right? So we can measure collaboration. I have metrics on collaboration and um, say how are we collaborating? But there’s a difference between, well, I’m gonna meet those metrics by [00:33:00] making sure that every Friday every group presents to a different group about what they’ve been doing and like their key challenges versus pair programming day to day. Right? There are two set. There’s one system that says you need a pair program day to day, and the other system defines collaboration differently and defines the interactions around that differently, but like they could still lead to kind of the same KPI movement. That’s an example like that I’m currently thinking about. But one example where I think like we can get to what healthy looks like, but there’s a number of different ways to make those symptoms click. To your example about the person in the bathtub, right, and it’s the system that drives that.
When you understand that, oh, body temperature isn’t only controlled through the temperature of the water that in which they sit, but like there’s a bunch of other interactions. That’s coming from the system and
Andy: So we can, we can measure all these points, but we always know that that’s not the complete picture. [00:34:00] There’s a a system and the system itself, the system you are choosing has a lot of effects that you just can’t tell the difference of, to put this in like software terms, my tests can show the presence of a bug,
Mon-Chaio: mm-hmm.
Andy: like my standards and the measures, the KPIs can tell me the presence of a bug, but hitting all of my, all of that or passing all of my tests does not prove the absence of a bug.
Mon-Chaio: Absolutely. Absolutely
Andy: what that whole system is really matters. To make it a robust, healthy organization.
Mon-Chaio: right. I like that example. I thought you were gonna go in a different direction. I do like that example. I thought you were gonna bring back the example you gave a few episodes ago, which is you may be able to fix the bug, but without under understanding the system, you’re never gonna know, is that bug gonna come back, or is a similar bug gonna come back, or what’s the likelihood that I’m gonna produce a [00:35:00] bug tomorrow?
Andy: Yeah. And, and, and I think that one also works as well. I, I was going for this because it’s this idea that your, your standards are never complete.
And so, and so, as you were saying, that system, the system that you choose since, as we were saying, there’s a, a kind of fixed component of of the human behavioral psychology, which is for all intents and purposes, let’s take that as fixed. And then there’s the malleable component of it, which is what are the policies and procedures and, and, and methodology that we’re using within this group. And you can do like an infinite number of that one and an infinite number of them will produce that KPI hitting what you want it to
Mon-Chaio: Mm-hmm.
Andy: but not, not all of infinite number will produce something that [00:36:00] everyone would look at and say, oh, that’s a healthy system.
Mon-Chaio: Right. Or even that, that’s a coherent system, nevermind, healthy or not.
Andy: Yeah,
Mon-Chaio: but I, I actually think that in order to get to healthy reliably, this again gets to the repeatable of it. You need to have a system that’s coherent, and I think so many people’s systems are incoherent. They’ll say one thing here and then another policy.
You’ll be like, but that, doesn’t that go against what you were talking about there? And they’ll say, no, no, no. That’s just a, a, a process change over here. A good example that I think about a lot is collaborative, which is individualized work. You’re like, no, we want to be one company and think and solve problems as one company.
Okay, so why did you give that engineer one thing and they’re not allowed to touch this other thing and they need to sit there and think about this one thing? Oh, well, because like, we need to be able to like, have owners for stuff. Okay, but
Andy: have to own it. Otherwise, how am I ever gonna get anything
Mon-Chaio: right. Okay. But then, you know, so that becomes an incoherent system.[00:37:00]
Andy: Yeah.
Mon-Chaio: Right. And I think a lot of people live in incoherence without knowing it, because they don’t think in terms of systems to those, to them, those two things are disconnected entities.
Andy: Yeah. And to take this kind of, I think the code analogy can be useful here, uh, quite a bit to take this, uh, to that. An incoherent system is a system full of bugs. Code has bugs when two different parts of it have different understandings you could say of how everything works. Which also means that that’s the exact same code that your solutions, your interactions to make changes to it have unanticipated, unanticipatable outcomes.
And so if you have an incoherent system, then anything you select as a solution. You won’t even be able to figure out what’s gonna happen because those interactions are now going [00:38:00] to be so far out of the realm of what’s controlled within that system that you, you can do that, you can make a change there, but who knows what’s gonna happen out of it.
Mon-Chaio: Yeah, no, absolutely. I like that. I, I think that makes a lot of sense in my mind. I think the challenge here for me is, and maybe this is just because I like thinking about this things this way. The challenge here isn’t so much around the standards and solutions and symptoms. I think there are big challenges there that we could spend entire episodes or multiple episodes talking about, but to me it’s like getting the systems right and I’m not really sure if we have a tactic for what is a good system or good set of systems that you can choose between for technical organizations. I think the paper itself says that there’s a lot more research that needs to be done into like what a quote unquote good system [00:39:00] is, if there is even such a thing. But if we constrain it down to technical organizations like do we have any guidance? I mean we can’t present. Like we’re not gonna come on this podcast Andy, and be like, okay, here’s the four systems you should aim for that we know are systems that are good. Just probably ’cause we haven’t thought about it enough. Probably. ’cause maybe we need to come up with ’em.
I don’t know. Right. But like in the absence of that, like do we have any tactics around the system’s knowledge sphere?
Andy: I would say, given what we’ve talked about is.
I think it’s, it’s focus a lot on the coherence of the system that you have.
Mon-Chaio: Mm-hmm.
Andy: Are you being inconsistent on things? Now, it may be that you need to be inconsistent, but that inconsistency should have a consistency to it.
Mon-Chaio: Right.
Andy: There should be like a clear understanding about like why this one needs to work this way and this part needs to work this way.
That that’s, that’s clearly reasoned and has meaning to it. [00:40:00] So I would say that’s the tactic that we’ve come up with out of this is one, absolutely, figure out those KPIs. Understand what to you at the moment good looks like, understand as well that that’s gonna keep evolving as like you learn more about how things happen. Just like in medicine the number of measurements that we can do on the human body to say that this is what healthy is, just keeps growing and growing and growing and changing as we get a better understanding of the underlying system to say like, oh yeah, we thought that meant healthy, but actually it only meant healthy in about 80% of the time, and the rest of the time we actually made things much worse. So this is, this is a better understanding of what’s going on.
Mon-Chaio: Yeah, no, I agree with that. I think, um. Whatever system you’re using, make it more, more, more coherent. And then just engage in systems- level thinking. Like stop spending so much time in your standards and solution space and spend more [00:41:00] time thinking about your system space.
Andy: Yeah.
Mon-Chaio: Right. Um, and then the, the other thing I’ll offer from this paper is just that I really like thinking about things in terms of these four knowledge spheres.
So I would say before you get into any diagnosis. Think about what your knowledge spheres are that are gonna contribute to your diagnosis. Make sure you’re not missing anything blatant. Uh, make sure you do your research if you need to, and then diagnose versus just say, well, I have a symptom. Now I need diagnosis to jump to solutions without thinking about the other two spheres.
Andy: Yeah. Yeah. I think that’s a really good one, is make sure that you’ve got, you’ve gone through all of these a bit like we had with the six box model, where it was like, make sure you’ve thought through these different areas. This one is the four box
Mon-Chaio: right.
Andy: Don’t jump to any conclusions. Don’t start doing anything until you’ve at least given some thought to what’s going on in each one of these.
Well, Mon-Chaio, I think we have [00:42:00] talked this less than 26 page paper to death. There’s probably more we could talk about, but we’ve left our listeners with a few nuggets of wisdom of what to think about, uh, the systems, making sure that they’ve talked, they’ve thought through all four areas of knowledge, spheres of knowledge, and uh, yeah, hopefully they can go out and diagnose. We’re gonna have to figure out where we go next with this. Is it to what are different systems that we have available to us? Is it to how could you set standards? Is it to what symptoms look like in a tech organization? We will find out next time on the TTL podcast. If you’d like us to help you with understanding your system, understanding your symptoms, understanding your solution space that you have available to you, or understanding what a standard might look get in touch.
We’re both available at [00:43:00] hosts@thettlpodcast.com. And until next time, Mon-Chaio, be kind and stay curious.
Leave a Reply