Make New Maps

Christopher Columbus wasn’t just a terrible human and an agent of colonialism of a massive scale, he was also on the wrong track completely. When he set sail in 1492 he wanted to find a new route to the Indies, to bring back precious spices. He had a map and he thought he had a route. That is why he sailed pretty much in a straight line due West from Spain convinced that by following the map he had he would find a new route to a well known destination.

At around the same time another, less known European explorer Amerigo Vespucci realised that what he had stumbled across was not depicted on existing maps; it was a whole “new world” (from his no less eurocentric and colonial perspective). Vespucci’s journeys fanned out over large areas as he followed the coast to see what was there. Vespucci was not finding a new route, he was creating a new map and discovering potential new destinations.

Shifting a system requires us to make new maps to make sense of the journey, as it unfolds, reading the shifting landscape, learning from one another’s reports, deciding on possible new destinations and correcting our course. As Jess Dart the founder of Clear Horizon put it on Day Three of our Learning Festival: you have to move away from being accountable to predetermined targets and instead “learn your way through change” including where you should be headed.

To understand how to “learn your way through change” we brought together innovative evaluators from around the world, to share their insights, methods and approaches. At the core of our discussions were a set of common challenges involved in shifting entire systems. 

System change is a highly collaborative process, with many players making complementary investments over long periods. How can these players, each with different perspectives, share and synthesise what they are learning to make sense of the direction and pace of change especially when initiatives might spend a long time gestating in the margins before they gain traction?

By whose yardsticks is progress to be measured? Who decides on what the destination should be? Bonnie Chiu challenged us to reflect on whose voice is being heard, whose perspective is being attended to.

How can we spot early signs that systems systems are shifting, when a minimum set of components for an alternative system are coming into focus: what we call a minimum viable system? 

Complex systems are never linear: there is rarely a neat correlation between a change in an input and an output. Change is usually emergent and hard to predict. How can we see the shape that change is taking, while also understanding the deeper underlying conditions which determine the shape the system takes?

Across a day of discussion among leading innovators in the field of evaluation five lessons stood out. 

Respect complexity; don’t be engulfed by it.

Don’t get lost in the vast detail of systems. Learn to step back; take in the system from different vantage points; look for patterns. Focus on what Beth Smith of the Cynefin Group calls “the vector of change”; its direction and pace. Evaluators have to help people see these bigger pictures, rather than just counting outputs.

Angie Tangaere, from the Southern Initiative in New Zealand, described one way to sense the bigger picture, rooted in Indigenous knowledge systems, moving “away from planning, programming, service delivery, towards ‘attuning’: attuning to the planet, attuning to each other, attuning to rhythm, attuning to what space looks like and attuning to relationships as a fundamentally different way of being.” Making sense of how a system is changing means sensing dissonance, when the system is failing, and resonance, when it is allowing different people to come together to achieve their aspirations. 

Making sense, going forward.

Evaluators cannot just deal in the past, telling us what happened, how the dots-joined-up backwards, sometimes long after the event. They should help people interpret what might happen, what might be coming and so how to act to influence the path that change takes. 

To open up the potential for future systems, evaluators need to help innovators understand the multiple pathways along which change could move, according to Emily Gates, from Boston College, and the different signals which show in which direction change is moving, so they are primed to learn. 

Create value before you judge it.

Evaluators are usually brought in to deliver a judgement at the end of a programme, to tell us what impact a programme had and whether it achieved its goals. That isn’t much help if you are trying to work out what your next step should be while learning in real time as a system shifts. In system change evaluators need to be engaged in the innovation process, to help guide it, without compromising their ability to offer independent and sometimes critical feedback. 

Evaluation should encourage conversation.

Evaluators of system change cannot come along with their clipboards, at the end of a programme, to count inputs and outputs. Their role should be convivial: to encourage informed, thoughtful conversations, using appropriate evidence, to help people involved in change make sense of it. 

That means having different kinds of conversation depending on what you are trying to learn. Anne Bergvith Sørensen, from Homes for All talked about the need for different approaches to learning at different levels of the system, in her case the systems that address homelessness. Conventional programme evaluation can work for innovations at the micro level, often yielding technical feedback on how to make things work more effectively. But at the macro level, where the Alliance is working to change the national discourse around homelessness, it is a case of “trying to track changes in what is said and what is written about homelessness and create a new story, a new landscape.” 

At the Omidyar Foundation, Rob Riciliagno has shifted evaluation away from an episodic assessment of impact towards near constant learning, about the conditions which determine the health of a system and the actions the Foundation could take to help shift the system in the context of all the other factors in play. Honing an intervention to be more effective, is like finding a new route on a map; understanding what might make an entire system healthier is much more like creating a new map. 

In any complex system, with many interacting parts, creating a new map of possibilities has to be a highly collaborative activity. 

Beth Smith, at the Cynefin Company, uses mass storytelling, to allow people en masse within a system to tell their story, directly, to one another, to surface hunches and underlying beliefs. Sense making across a system, Smith says, required highly distributed learning and cognition as well as interpretation: “Sense-making becomes a social process, creating pictures of change across different levels from the individual, through the community to the national.” 

Shifting context, changes what’s possible

It is context that casts us as powerful or powerless. In some contexts we are powerful, in others powerless. To make deeper system change it is not enough to fund people to try to act differently in a given context, when the odds are stacked against them. We need to fund people who are shifting the context in which new kinds of work, care, learning, building become possible. We know how to count what people do and what they deliver. That’s the stock in trade of grant applications to foundations: a theory of change, a set of outputs. Yet if what really matters is how we change the context for what can be done, what Kenneth Bailey, from Design Studio for Social Intervention, calls the arrangements we step into, then we need a quite different approach to thinking of how value is created. Shifting how we think of evaluation, who holds whom to account for what, is part and parcel of that change.