Sense-making through stories: an introduction to Most Significant Change

As the evaluation space becomes increasingly more technical, digital and complex – think big data, impact dashboards and collective impact – the value of personal stories of impact can sometimes be overlooked. After all, could a first-person narrative really offer more insight on how an initiative is performing than a real-time collation of hundreds of digital survey responses?

In some cases, yes. Especially if that story was collected and selected using the Most Significant Change technique – an evaluation method specifically designed to uncover deeper and potentially unexpected program impacts. But before we delve further into the powerful potential of storytelling for evaluation insights, let’s first explore the Most Significant Change technique.

What is Most Significant Change (MSC)?
MSC is a participatory, qualitative evaluation tool involving the collection and selection of first-person stories. Each testimony must contain a beginning, middle and ending. This format is important and incredibly effective – every human culture in the world uses storytelling as a vehicle for learning and improvement because our brains are wired to make sense of information through stories. Put simply: you are more likely to remember and act on information shared in a story format.

A typical MSC story might start with a bit of information about the storyteller – usually a beneficiary, program staff member or other key stakeholder – about who are they and what things were like for them before they became involved with the initiative. It then describes how the person came to be involved in the initiative and conclude with a description of what changed for them and why the changes were significant to them.

Story selection
With MSC, the process of story selection is just as important as story collection, and is where data validation and analysis occurs. Key stakeholders are brought together to read the range of stories collected and asked to decide which contains the most significant change that occurred as a result of the program/initiative. The ensuing conversations about why each stakeholder selected their particular story contain rich information about the program and delivers real insight into what stakeholders value.

How is MSC different to other approaches?
MSC is often compared to case studies and appreciative inquiry. Although it can be similar in appearance to a case study, the methodology is quite different with unique steps designed to help organisations learn and improve that go beyond what stand-alone case studies can offer. For instance, case studies are often selected by program staff alone, whereas MSC relies on panels of people and a participatory process to decide what stories are important and should be elevated. The heart of MSC is that it elicits value from the individual in the story and from the group selecting the story.

How is MSC different to other approaches?
MSC is a great companion or supplementary approach to many traditional evaluation approaches, and is particularly useful where:

  • hearing the voices and lived experience of participants and beneficiaries is important
  • the program is decentralised and there are diverse outcomes and indicators
  • the changes are intangible
  • a program is still unfolding or emerging
  • the environment is changing rapidly
  • there is a desire to involve stakeholders in making judgements about data in a really engaging manner
  • the program itself is highly participatory.

When isn’t MSC Suitable?
MSC is potentially unsuitable when 

  • there is a need to understand the scope of the change 
  • you want to prove the changes happened as a direct result of your program 
  • the program is highly contentious or there is a lot of emotion around it in the community 
  • you just want some good news stories or marketing material (hire a journalist – it will be cheaper!) 

With this in mind, let’s look at a couple of examples where we’ve used MSC to help organisations uncover valuable insights that may have otherwise been missed. 

Equal Playing Field
Equal Playing Field is a Papua New Guinea-based NGO who run respectful relationship programs for young people. The programs are delivered by volunteers (young university students) to high school students. Volunteers and high school students were asked to provide their stories through an MSC process as a powerful way to capture experiences of the program. The stories revealed that prior to the program, participants had no idea their behaviour would be considered teasing. This had significant impacts for both data collection and overall understanding of how the program was working. For example, survey question used in program evaluation asked participants to respond to the statement “I think it is funny to tease people”, with the program managers expecting to see a reduction in those endorsing this statement by the end of the program. While the participants were now aware of teasing and longer thought it was funny, the existing survey wasn’t sensitive enough to capture this changeThe use of MSC informed a redesign of the survey to provide more accurate data about the program’s real impact 

Target Ten Dairy Extension Project
The purpose of this project was to increase the milk production of dairy cows in GippslandWe expected to hear stories of change about cow nutrition and paddocks, what MSC elicited was a multitude of stories about women’s empowerment and other unexpected themes. For instance, we discovered that many women running the farms actually didn’t want to be on them anymore, and that they intended to leave dairy farming entirely. The project managers were then faced with the question of whether to incorporate these discoveries into their theory of change and make it part of their project design or to ignore these findings and push away from them. They (very sensibly in our books) decided pivot and focus their efforts on women running farms and succession planning.  

These examples illustrate how MSC can help strengthen and improve data-gathering for evaluation and provide space to explore the complex and unexpected impacts of a program initiative.

If you’d like to like to learn more, our next Most Significant Change course starts April 22 2021. Designed and delivered by Dr Jess Dart, who co-wrote the Most Significant Change User Guide with MSC creator Dr Rick Davies, this course will provide you with the practical tools and resources you need to undertake MSC in your organisation.

Leave a Reply

avatar
  Subscribe  
Notify of

CLEAR HORIZON ACADEMY

  • (+61) 3 9425 7777
  • Email us
  • 132B Gwynne Street, Cremorne, 3121, VIC

Sign up to our Newsletter

The Clear Horizon Academy is a learning initiative of Clear Horizon.

We acknowledge the Traditional Custodians of the lands on which Clear Horizon is located and where we conduct our business. We pay our respects to Elders, past, present and emerging. Clear Horizon is deeply committed to honouring Indigenous peoples’ unique cultural and spiritual relationships to the land, waters and seas; and to transforming evaluation practice to uphold and reflect Indigenous values.
Join Waitlist We will inform you when the product arrives in stock. Please leave your valid email address below.
Jess Dart sq

Dr Jess Dart

CEO & Founder

Inventor of practical methodologies and highly demanded facilitator, Jess navigates complexity with comfort and helps her clients to become clear about their desired outcomes and how to get there.

Recipient of the 2018 AES Award for outstanding contribution to evaluation, Jess has over 25 years experience in evaluating and designing social change initiatives and strategies in Australia and overseas.

In October 2005, Jess founded Clear Horizon Consulting and is now CEO. She is also a board member of the Australasian Evaluation society.

Jess is passionate about developing and designing real world evaluation and strategy for social justice and sustainability. She particularly works with systems change interventions, large scale strategy and social innovation. After completing her PhD she co-authored the Most Significant Change (MSC) guide alongside Dr Rick Davies, which is now translated into 12 different languages.

MSC is well suited to evaluation in the complex emergent context. The latest innovation by Jess, Collaborative Outcomes Reporting (COR), is a collaborative form of impact evaluation.

Jess is also an active mum and has two teenage boys. In a quiet moment, she loves reading far-future science fiction and enjoys long distance running.

Please provide your information
before downloading pdf file