Why I don't believe in Evidence-Based Management (1 of 6): What to expect

Tags: Agile

In an ancient Indian folktale a group of blind men use their hands to discover what's standing in their midst. For a bystander it's obviously an elephant. But because each of the men feels a different part of the elephant, they all arrive at different conclusions. The man holding the tail is certain that it's a big rope. But the man touching the tough hide is convinced that it's a wall. But feeling the tusks convinces another man that it's a spear. The men continue to bicker all night long, unknowing that they are all partially right and partially wrong.

Evidence-Based Management (EBM) is an emerging movement that aims to improve the quality of decision-making by urging managers to use 'the best available evidence' to support their decisions. In a sense, it involves the application of scientific principles and empiricism to management decisions (Pfeffer & Sutton, 2006).

Recently, Scrum.org has picked up on EBM and is now actively promoting it as the next logical step in improving organizational agility within the context of software development. Although I will introduce EBM more thoroughly in my next post, EBM combats the ‘snake oil’ solutions, just-in-time consultants and management hypes that permeate the management literature and industry. Although I applaud Scrum.org for trying to push organizations to the next level, I'm not sure if EBM is really the way to go. I will use this series of posts to explain why and - hopefully - contribute to a more workable approach.

But before diving into this nitty gritty of it all, it may help to know why I am passionate about this topic. I am a Scrum Master and have been working in Agile software development for several years. But I also frequently double as a data analist or researcher for research papers, statistical analyses and reports intended to support decision-makers. Maybe it's exactly this experience that makes me skeptical of EBM. Although I believe that it can broaden the perspective of decision-makers beyond politics, gut feeling, managements hypes, fears and hopes and sensitize them to the quality of the information they use, I am worried that the limitations of EBM are not well understood nor taken into account. In a way, I am worried that EBM will become exactly the 'snake-oil' solution that it's trying to prevent.

I will use the following posts to hammer out my objections in more detail. But I thought it'd be useful to give you an overview of what to expect:

  • My first objection is that EBM suffers from a considerable epistemological problem in how evidence is being classified as either ‘direct’ (big ‘E’) or ‘circumstantial’ (small ‘e’) evidence. The former is weak, and should be not be given too much weight in decisions, while the latter is supposedly an unambiguous source of ‘irrefutable evidence to prove an assertion’. In my third post, I will show that this distinction is problematic because there is no practical method of gathering evidence in organizational research that even comes close to being ‘direct’, ‘unambiguous’ or ‘irrefutable’. And if all evidence is circumstantial and weak at best, what exactly is the value of EBM?
  • My second objection follows from the first in that this problematic definition of evidence opens the door for manipulation. Pretty much everything can be proven with data, especially if this data was collected and analyzed with limited scientific training. In my fourth post l will argue that EBM can be easily used (purposely or not) to falsely present evidence as being ‘irrefutable’, stifle resistance and debate, and push personal agendas under the guise of the ‘scientific method’.
  • My third objection is that EBM (as a whole) sets very ambitious goals for managers. It may sound nice on paper, but in my fifth post I will argue that it’s unlikely that managers will be able to apply EBM principles in their day-to-day job, considering the required skills, time and resources. Chances are this will be delegated to (expensive) consultants, and doesn’t that defeat the purpose of EBM as a set of principles to be applied by managers themselves?

In my next post, I will define EBM and discuss how it is being applied by Scrum.org for software development. I will use my final post to summarize and offer a more reasonable, practical and sensible approach that addresses my objections. It is my hope that this series will spark some much-needed debate on this topic, as I've seen very little of that so far. In any case, I will try to support my assertions with as much as evidence as I can gather. I want to avoid this being too much of an opinion piece :)

References

Pfeffer, J. & Sutton, R. I. (2006). Hard Facts, Dangerous Half-Trutsh and Total Nonsense: Profiting from Evidence-Based Medicine. Boston, MA: Harvard Business School Press.

Scrum.org (2014). Empirical management explored: Evidence-Based Management for Software Organizations. Retrieved August 9, 2014 from https://www.scrum.org/Portals/0/Documents/Community%20Work/Empirical-Management-Explored.pdf

Christiaan Verwijs
Christiaan Verwijs

Scrum Master, Trainer, Developer & founder of Agilistic