In some cases, yes. Especially if that story was collected and selected using the Most Significant Change technique – an evaluation method specifically designed to uncover deeper and potentially unexpected program impacts. But before we delve further into the powerful potential of storytelling for evaluation insights, let’s first explore the Most Significant Change technique.
What is Most Significant Change (MSC)?
MSC is a participatory, qualitative evaluation tool involving the collection and selection of first-person stories. Each testimony must contain a beginning, middle and ending. This format is important and incredibly effective – every human culture in the world uses storytelling as a vehicle for learning and improvement because our brains are wired to make sense of information through stories. Put simply: you are more likely to remember and act on information shared in a story format.
A typical MSC story might start with a bit of information about the storyteller – usually a beneficiary, program staff member or other key stakeholder – about who are they and what things were like for them before they became involved with the initiative. It then describes how the person came to be involved in the initiative and conclude with a description of what changed for them and why the changes were significant to them.
With MSC, the process of story selection is just as important as story collection, and is where data validation and analysis occurs. Key stakeholders are brought together to read the range of stories collected and asked to decide which contains the most significant change that occurred as a result of the program/initiative. The ensuing conversations about why each stakeholder selected their particular story contain rich information about the program and delivers real insight into what stakeholders value.
How is MSC different to other approaches?
MSC is often compared to case studies and appreciative inquiry. Although it can be similar in appearance to a case study, the methodology is quite different with unique steps designed to help organisations learn and improve that go beyond what stand-alone case studies can offer. For instance, case studies are often selected by program staff alone, whereas MSC relies on panels of people and a participatory process to decide what stories are important and should be elevated. The heart of MSC is that it elicits value from the individual in the story and from the group selecting the story.
How is MSC different to other approaches?
MSC is a great companion or supplementary approach to many traditional evaluation approaches, and is particularly useful where:
- hearing the voices and lived experience of participants and beneficiaries is important
- the program is decentralised and there are diverse outcomes and indicators
- the changes are intangible
- a program is still unfolding or emerging
- the environment is changing rapidly
- there is a desire to involve stakeholders in making judgements about data in a really engaging manner
- the program itself is highly participatory.
When isn’t MSC Suitable?
MSC is potentially unsuitable when:
- there is a need to understand the scope of the change
- you want to prove the changes happened as a direct result of your program
- the program is highly contentious or there is a lot of emotion around it in the community
- you just want some good news stories or marketing material (hire a journalist – it will be cheaper!)
With this in mind, let’s look at a couple of examples where we’ve used MSC to help organisations uncover valuable insights that may have otherwise been missed.
Equal Playing Field
Equal Playing Field is a Papua New Guinea-based NGO who run respectful relationship programs for young people. The programs are delivered by volunteers (young university students) to high school students. Volunteers and high school students were asked to provide their stories through an MSC process as a powerful way to capture experiences of the program. The stories revealed that prior to the program, participants had no idea their behaviour would be considered “teasing”. This had significant impacts for both data collection and overall understanding of how the program was working. For example, a survey question used in program evaluation asked participants to respond to the statement “I think it is funny to tease people”, with the program managers expecting to see a reduction in those endorsing this statement by the end of the program. While the participants were now aware of teasing and longer thought it was funny, the existing survey wasn’t sensitive enough to capture this change. The use of MSC informed a redesign of the survey to provide more accurate data about the program’s real impact.
Target Ten Dairy Extension Project
The purpose of this project was to increase the milk production of dairy cows in Gippsland. We expected to hear stories of change about cow nutrition and paddocks, what MSC elicited was a multitude of stories about women’s empowerment and other unexpected themes. For instance, we discovered that many women running the farms actually didn’t want to be on them anymore, and that they intended to leave dairy farming entirely. The project managers were then faced with the question of whether to incorporate these discoveries into their theory of change and make it part of their project design or to ignore these findings and push away from them. They (very sensibly in our books) decided pivot and focus their efforts on women running farms and succession planning.
These examples illustrate how MSC can help strengthen and improve data-gathering for evaluation and provide space to explore the complex and unexpected impacts of a program initiative.
If you’d like to like to learn more, our next Most Significant Change course starts April 22 2021. Designed and delivered by Dr Jess Dart, who co-wrote the Most Significant Change User Guide with MSC creator Dr Rick Davies, this course will provide you with the practical tools and resources you need to undertake MSC in your organisation.