Self-Organization As A Survival Skill

In the new Scrum Guide, Scrum Teams are no longer described as “self-organizing”, but as “self-managing” instead. This distinction may seem trivial, but it helps understand two essential truths about Scrum that we explore in more detail in this blogpost. The first is how Scrum uses self-organization to act as a lever to make organizations more agile. The second is how Scrum Teams require a high degree of self-management to make this happen.

This blog post is an excerpt from a more lengthy chapter in our book the Zombie Scrum Survival Guide. If you prefer to listen to this post, you can do so with this episode of our podcast.

What Is Self-Organization?

In different scientific domains, from biology to sociology and from computational sciences to physics, self-organization is the process by which order arises spontaneously from something that is initially disorganized. This order is the result of self-organization only when it emerges from the interactions of the smallest units of the system and isn’t imposed by outside influences. Self-organization happens all around us and on many different levels. It happens when the wind creates beautiful shapes in the sand. It happens when ants work together to build massive colonies, without a clear intelligence guiding it. And it happens when people effortlessly avoid walking into each other when large crowds intersect.

“Self-organization is the process by which order arises spontaneously from something that is initially disorganized.”

A good example of this is when you bring a large group of employees together. Initially, there will be chaos as they don’t know how to work together. Managers can create order by giving instructions. But since this order is imposed, it isn’t self-organization. Alternatively, employees can come to a shared understanding of how to perform and coordinate their work without direction from outside. Although this example uses “employees” as its smallest unit, you can replace them with Scrum Teams for the same effect. There too, rules, structures, and collaborations will form spontaneously when you put fifty teams together. Whether they are also effective is another question though.

The success of self-organization, and whether or not it turns into chaos or useful solutions, depends on two major ingredients. The first are the simple rules the teams follow. And the second is the autonomy they actually have.

Self-Organization Through Simple Rules

The first ingredient for successful self-organization is the simplicity and quality of the rules that are followed by the smallest units of a system. A common example of this is the flock of birds that creates elaborate and complex patterns in the sky called a murmuration. For all its beauty, these patterns are the result of a few simple rules that all birds adhere to: they maintain the same speed and stay at a similar distance from a handful of birds close to them. As each individual bird follows these simple rules, tiny variations in speed, distance, and direction cause big changes as the flock rapidly elongates, turns, and flips. Without these simple rules, chaos would ensue. Self-organization does not reflect the autonomy of individual birds, but rather how group-level patterns spontaneously emerge when individual members of a group follow a few simple rules.

The Scrum framework purposefully defines one essential rule for Scrum Teams to follow: deliver a Done Increment every Sprint that achieves the Sprint Goal. This Increment is the primary driver of transparency, inspection, and adaptation. It gives purpose to all the structural elements that make up the Scrum Framework; its roles, artifacts, and events. Although following this rule is certainly not as simple as maintaining the same speed and distance from the birds around you, maintaining it will cause system-wide changes.

The people who build the product — the Scrum Teams — will discover things that get in their way of releasing a Done Increment every Sprint. They may discover that they lack skills or depend on people outside the team to do work for them. Or a lack of mandate makes it hard for Product Owners to define a clear Sprint Goal. As Scrum Teams identify and remove impediments it becomes progressively easier to conform to the single rule. This enables them to improve their way of working more quickly in response to the increasing amount of feedback they are getting on their work, and the speed at which they are obtaining this feedback. In other words; they are becoming increasingly flexible and agile to their environment. The Scrum framework acts as a lever for system-level change by having Scrum Teams focus on releasing a Done Increment every Sprint.

Unfortunately, teams that suffer from Zombie Scrum are either unwilling or unable to follow this single rule. Here, the lever doesn’t work and self-organization doesn’t happen, or not in a direction that matters to agility.

“The Scrum framework acts as a lever for system-level change by having Scrum Teams focus on releasing a Done Increment every Sprint.”

Self-Organization Through Self-Management

The second ingredient for successful self-organization lies in the autonomy that people and teams have to determine their own rules. One way to think about this is to consider the work that a team does as a river. An impediment or challenge may appear in the form of a rock that is put in its path. The more constrained the river is, the fewer options there are to flow around the rock. Increased autonomy gives teams the ability to allow their work to flow around rocks that get in their way.

Organizational scientists often describe this as “self-management”. In this philosophy on management, teams are responsible for a complete product, an isolated part of a product, or a particular service. Instead of having a manager who decides for them, or having to adhere to strict policies and protocols, teams have a degree of autonomy in areas like:

• How new members for a team are selected and recruited.
• How teams and its members are rewarded and evaluated.
• How teams create a safe and collaborative environment.
• How teams are trained in important skills, and by whom.
• How teams spend their time.
• How teams synchronize their work with other teams, departments and units.
• How teams set goals.
• What facilities and tools teams need to do their work.
• How decisions are made in the team.
• How teams distribute their work.
• Which methods, practices, and techniques teams use.

For each of these areas, the autonomy of a team ends up somewhere between “No autonomy at all” and “Full autonomy”.

The concept of self-managed teams might seem novel, but it has been around for a long time. Self-management is an important part of the socio-technical systems approach (STS) that was developed by the Tavistock Institute of Human Relations during the Second World War. As a result of this work, self-managed teams started appearing everywhere, including in many car manufacturing plants. Instead of the traditional assembly-line manufacturing that had previously prevailed, teams took responsibility for the completion of entire sub-systems of a car (the brakes, electronics, etc.). Teams were also responsible for their own planning, scheduling, task allocation, recruitment, and training — without involvement from management. The extensive research that has been done on socio-technical systems over the years shows a huge boost in job satisfaction, motivation, productivity, and quality. The Toyota Production System (TPS) that would later inspire the Scrum Framework and Lean methodology is an example of such a socio-technical system.

While the Scrum Guide used to Scrum Teams as being “self-organizing”, it always meant for them to be “self-managing” in order for the process of “self-organization” to happen. Scrum Teams have all the roles and responsibilities needed to make decisions about their product and how to do their work. In reality, most Scrum Teams are severely limited in their ability to self-manage, however. In an attempt to reduce the potential chaos and disorder that they expect will happen when teams self-manage, many organizations tightly control how teams do their work instead. They either don’t understand the mechanisms of self-organization or don’t trust the outcomes. With Zombie Scrum as a result.

Self-organization Is A Survival Skill In A Complex World

Complex environments are characterized by high degrees of unpredictability and uncertainty. This makes them volatile and rife with risk. Markets shift in the blink of an eye, new technologies achieve widespread popularity seemingly overnight, and as they do they may be found to harbor security vulnerabilities that need to be fixed immediately. New competitors enter the market with a superior product, undermining seemingly unassailable market positions. And then there are global catastrophes, like the financial crisis of 2008 and COVID-19, that upend economies overnight and take companies entirely by surprise. As our world becomes increasingly globalized and interconnected, so does the chance of unpredictable and highly impactful events that demand immediate adaptation. The statistician Nassim Taleb calls these events “Black Swans”.

Taleb goes on to describe how organizations often optimize for what he calls “Robustness”. In an effort to try to reduce volatility, they rely on standardization and centralized coordination to reduce harmful variation both within and outside the organization. For example, all teams have to use the same technologies or follow the same procedures when solving specific problems. Or they create centralized steering committees to guide multi-team product development. By adopting rigid standards and coordination structures, organizations are able to limit the impact of variation when the changes are small. But in a world that is increasingly volatile, this rigidity prevents them from adapting to change, and can even break them entirely.

Another way is to optimize for “Antifragility”. Instead of trying to resist variation and shocks, antifragile systems grow stronger when they are pressured. For example, engineering teams at Netflix created a tool called “Chaos Monkey” to randomly terminate services in their infrastructure. Every time a terminated service ends up causing disruptions to end-users, engineering teams redesign the architecture to reduce the impact. Over time, responding to these kinds of random shocks helped Netflix make its infrastructure more resilient.

Space Exploration Technologies (SpaceX) has a launch cadence that is purposefully higher than other launch providers. Every time a launch fails, their self-managed teams update technology, protocols, and processes to avoid similar failures in the future. Other organizations, including Procter & Gamble, Facebook, and Toyota, run many small experiments at the same time to explore different alternatives. Although most fail, some strike gold. More importantly, their self-managed teams learn from failures and grow stronger because of it.

Three threads are apparent in antifragile organizations:

• They rely on self-managed teams to self-organize around problems as they appear.
• They encourage experimentation to grow stronger through failures.
• They spend effort to learn from failure through single- and double-loop learning.

Taken together, organizations develop the skills, technologies, and practices to not only survive the uncertainty of complexity, but to actually thrive on it as they can adapt faster than others. Unfortunately, the variation and redundancy that is necessary for antifragility is often seen as inefficient and wasteful by organizations where Zombie Scrum is flourishing.

The concept of antifragility ties together much of what Agility is all about. The Scrum framework actively promotes it. It relies on self-managed teams to self-organize around challenges that get in their way. By following the single rule of releasing a Done Increment every Sprint, everything that makes it hard for teams to do so becomes apparent, including many of the factors that optimize for robustness, but not antifragility, like rigid control structures, lack of mandates, long feedback loops, and highly specialized (but not distributed) skills. By releasing a Done Increment every Sprint, teams effectively introduce more opportunities for success and failure, giving them opportunities to reflect on their results and learn. When enough teams in an organization do this, the whole system becomes increasingly antifragile.

The Bottom Line

In his novel “Seveneves”, author Neil Stephenson describes a catastrophic event where a dense field of debris suddenly appears around the Earth and is about to rain down and eradicate all life. In an effort to save humanity, engineers start building a space station for a few thousand souls that can continue to orbit the Earth until it becomes habitable again. Instead of building one giant station, the engineers instead design a massive swarm of smaller and autonomous stations that can connect and disconnect as needed. With all the debris still orbiting the Earth, and the catastrophic result that even a tiny grain of debris would have, a single station would be too dangerous. Although each unit in the swarm is still susceptible to disaster, their smaller size makes it easier to avoid incoming debris. Furthermore, the loss of individual units doesn’t immediately threaten the survival of the swarm as a whole. As a whole, the swarm can now self-organize around impending disaster more effectively than a single space station can.

This is a great metaphor for what the Scrum framework tries to achieve. It aims to break down traditional structures where work is standardized and tightly controlled through centralized management in order to avoid risk and variation. Like the large space station in the metaphor, these structures work well in stable environments. But our world is increasingly complex, and increasingly filled with unexpected debris that may cause havoc. So instead, the Scrum Framework enables antifragility by making self-managing Scrum Teams the metaphorical swarm from this story. As a self-managed team, every Scrum Team adds variability and thereby survivability.

In our book, we offer 10 powerful experiments to spur on self-organization and expand the ability of Scrum Teams to self-manage. We explain each experiment step-by-step and help you prepare.