Key points:
- Set up the organisation to ‘sense’ its environment
- Treat internal teams (almost) as external providers
- Conway’s Law should shape our organisation design
- Promise Theory is a useful approach to organisational agility
I have been doing quite a bit of work recently with various organisations to help them develop new capabilities for building and evolving software-rich services. Part of this work involves thinking about the responsibilities of different teams and how these evolve. This blog post captures some thoughts and links on how the more successful organisations arrange themselves for agility and responsiveness.
Set up the organisation to ‘sense’ its environment
As someone who performs music regularly in groups (both orchestras and choirs) I really appreciate the need for simultaneously responding to others and performing my part to the best of my ability; it’s a dynamic that is really rewarding… if everyone follows the same approach in sensing where the music is going. I have also done research into insect vision, hearing perception, and visually-impaired musicians: what connects these is an awareness of the value of sensing the environment.
It’s therefore often surprising for me to work with organisations that are effectively “blind” to their environment: organisations that do not (or cannot) listen to customers, organisations that have almost no situational awareness in terms of market conditions, and organisations that are blind to how their projects are failures in all but name. Part of the problem with these organisations seems to be that the leaders in these organisations expect to plan everything many months or years in advance, with no expectation of responding to events. These organisations expect to be right (correct) in their up-front assessments, and do not build in any useful feedback mechanisms. In fact, most organisations would be better to assume that their up-front assessments are wrong (incorrect) and build in sensing mechanisms to continually correct and re-align the goals:
When we use Cybernetic control, we explicitly expect the “output” or result to be wrong, and build in a sensing mechanism (“feedback loop”) so we can detect the difference between our intention and the actual result. The output is likely to be wrong because we do not control the external environment and/or our internal model of the external world is not sufficiently accurate. This is effectively how a heating thermostat works to control the temperature in a home or office. By building in sensing mechanisms to our organisation – such as user research, real-time web metrics, Kanban metrics, and truthful project governance statistics – and then adjusting our plans and organisational trajectory to adapt to the sensed data, the organisation is able to respond to change more effectively.
Treat internal teams (almost) as external providers
Several companies that have maintained their responsiveness whilst scaling to large numbers of people have adopted an approach that treats internal teams almost as if they were external teams (the teams are limited to around 10 people to maximise engagement and minimize the “explosion” of communication). Each internal team has a well-defined set of responsibilities that it provides to other teams, together with a sense of when and under what conditions that service will be provided. If that set of responsibilities is coherent and viable, the way in which those responsibilities are fulfilled can be made opaque to other teams, leaving the service-providing team free to change the “how” of the service without affecting the service itself.
Digitally-infused companies like Supercell and Spotify use collections of small, autonomous, aligned teams (or “pods”, “tribes”, etc.) to evolve discrete services that together make up the organisation’s whole offering. Teams have the sense of providing a service to other teams, and bring into the team the necessary skills to help them do that; these teams tend to be “cross-functional”, with a mix of skills to enable them to make rapid progress on all key differentiating areas of their service or responsibility, leaning on the services provided by other teams for non-core aspects they need.
From http://blog.idonethis.com/cells-pods-squads-structure/ – we’re aiming for high alignment, high autonomy.
With effective, viable responsibility boundaries in place, some capabilities can easily be moved outside the organisation, because the “service boundary” has been defined and proven to work in practice. Switching from an internal supplier to an external supplier need not affect many (if any) other teams in the organisation, because the service boundary remains in place. Similarly, if an organisation currently replies on an external partner for one service, it can more easily bring that capability in-house if the service interface has been well-defined. The ideas of Promise Theory help with this (see next section).
Conway’s Law should shape our organisation design
I have recently begun to explore in more detail the dynamics of organisations, particularly (but not only) within the context of software development. In particular, over the past 12 months I have had the privilege to facilitate my workshop on Organisation Design for Effective Software Systems many times, with over 100 people in total between UK and DE. One thing that is clear from these sessions is that most people find the idea of “designing the organisation” to be profoundly strange, particularly when we introduce Conway’s Law as a constraint on organisation communication structure:
Any organization that designs a system (defined broadly) will produce a design whose structure is a copy of the organization’s communication structure.
Conway’s Law has in recent years been assessed by academic research. Although the forces behind Conway’s Law manifest somewhat differently in different contexts, it’s clear that – when considering a single organisation of over about 100 people – the effects are widespread. I have spoken about the effect of Conway’s Law on organisation design quite extensively; here is a set of slides from a talk in Berlin in 2017:
Crucially for organisations that are seeking to be responsive, McCormack and colleagues at Harvard Business School found that:
“…the organization’s governance structures, problem solving routines and communication patterns constrain the space in which it searches for new solutions.”
In practice, this means that if we want to avoid unnecessary constraints on innovation and agility, we need to consider the shape, size, and interrelation of different teams withing the organisation. This in turn means we need to establish effective, viable responsibility boundaries for teams to help encourage communication that works in the interest of the products & services being produced, minimizing the communication that would work against effective product / service development & operation.
Promise Theory – a promising (!) approach to organisational agility
I recently finished reading Designing Delivery by Jeff Sussna, a book that brings together design thinking, service management and cybernetics to present a fresh approach to building and running business services. In the book, Jeff Sussna recommends the emerging field or practice of Promise Theory as devised and explained by Mark Burgess in various books and articles including the excellent book Thinking in Promises.
The essence of Promise Theory is that for resilient, adaptive systems (including organisations – you *do* think of your organisation as a system, right?) we should avoid expecting to tell or compel each part or team to deliver something, but instead compose our system/organisation from more modular parts that each offer and take promises from other parts. Just as in common usage of the word, a promise is a commitment to do something, where that commitment might be broken (accidentally or deliberately). By anticipating that some promises may be broken, we force ourselves to make each part of the system more resilient and adaptive, better able to withstand upsets and shocks.
Another reason why promise theory seems like A Good Thing to me is that it helps us to consider what makes a sensible promise; we have to ask ourselves some important questions: “Can this promise be kept? Is it reasonable to ask a team or a component to keep that promise? What conditions would need to be in place for that promise to make sense?”. These questions explore the viability of responsibility boundaries, and in my experience, many problems with complicated systems (both software and organisational) arise from poorly-chosen responsibility boundaries.
If the responsibility boundaries in our system are well-chosen and map to viable promises, we’re better able to swap out modular parts for alternatives without affecting other parts of the system or organisation. In software terms, we have well-defined interfaces between components. This in turn means that if we need to bring a capability or component in-house (or move it externally and consume it “as a service”) we have insulated most parts of the organisation from this change: the promises / interfaces remain the same, yet the team or component providing the service can change.
The responsive organisation – engaged teams and viable promises
For an organisation to be able to innovate and respond to changes in its environment, it needs to set itself up to sense the outside (and internal) conditions, through rich metrics, tight feedback loops between development and production, and an acceptance that its assumptions are most likely wrong. Through feedback and a willingness to alter direction, the organisation can “self-steer” towards more appropriate targets.
Effective teams work much better than simple groups of individuals. For a team to be effective, however, it should be limited in size (around 10 people) and have both autonomy and a clear mission. By treating internal teams almost like an external service provider, we are forced to consider the conditions under which a certain set of responsibilities can viably be provided. In turn, this gives us the flexibility to move capability out of – or into – the organisation with minimal disruption to other parts of the organisation.
To avoid constraining our innovation and capability and responsiveness, we should take account of Conway’s Law, designing the intercommunication between teams to anticipate the architecture or shape of the product & service we are building.
Finally, we can use approaches such as Promise Theory to explore and establish viable responsibility boundaries between different parts of the organisation to give us a modular, decoupled structure that is adaptable and scalable.
These slides from a talk in 2016 cover in more detail some examples of changing the team responsibilities for increased responsiveness:
References
- MacCormack, Alan, Carliss Y. Baldwin, and John Rusnak. ‘Exploring the Duality Between Product and Organizational Architectures: A Test of the “Mirroring” Hypothesis’, 1 October 2012. http://www.hbs.edu/faculty/Pages/item.aspx?num=43260
- Cells, Pods, and Squads: The Future of Organizations is Small – by Janet Choi, 1 May 2014, http://blog.idonethis.com/cells-pods-squads-structure/
Smart Digital: At Conflux, we use our ‘smart digital’ approach to help organisations around the world to assess, evolve, and accelerate their digital strategies through: digitally-infused product management, Continuous Delivery, software operability, and training & skills. confluxdigital.net
Matthew Skelton 