Stories

"Just wanted to thank you for the [article] series “Evaluation Stories.”  Many of our agency’s grant-supported programs have wonderful results that are hard to measure in numbers.  I’ve struggled for years to find a way to include these subjective results in our grant evaluations.  These articles are providing sensible and do-able solutions—I’m looking forward to Part 3.  Susan Eliot has provided me a great service through these articles—thank you, Susan!"  --Kathy Bangasser, CFRE, Grants Officer, Lutheran Social Services of South Dakota

Why Stories 

Many nonprofits share the same dilemma: their best program outcomes are the hardest to measure.

Often, the best illustration of a program’s intangible results is a compelling story. Stories reveal the faces behind the numbers and provide a context for making sense of the statistics. In addition, stories are good for capturing unexpected and indirect consequences. Touching stories can also poignantly convey a program’s needs or impacts to potential donors, the media, and the public.

We can gather stories in a variety of ways but the most common is through individual interviews. We may also encourage individuals to engage in personal reflection and journaling, in which case storytellers pass pre-written stories on to us for selection and editing. Stories are usually most useful when part of a traditional mixed-method evaluation, mixing story gathering with other evaluation techniques.

What is an Evaluation Story 

“An evaluation story is a brief narrative account of someone’s experience with a program, event, or activity that is collected using sound research methods,” writes Dr. Krueger. Five key factors differentiate an evaluation story from other stories. “The evaluation story,” he says:

• Is a deliberate, planned effort using systematic procedures.
•  Identifies the source of each story.
• Verifies stories with the storyteller or others familiar with the story.
• Includes a description of how stories were captured/handled using accepted research protocol.
• Includes a statement by the evaluator about the degree to which the story represents other individuals with similar circumstances.

In order to have a maximum impact on their audience, evaluation stories must be written up like other stories, not like research reports. Dr. Richard Krueger defines eight elements all masterful stories possess:

• Stories are about a person, not an organization.
• Stories have a hero, an obstacle, a struggle, and a resolution.
• One must set the stage for the story.
• The story must be allowed to unfold.
• It’s important to describe emotions.
• Dialogue adds richness to the story.
• Suspense and surprise add interest.
• Stories should reveal a key message.

 

Using Stories in Program Evaluation

 

Besides their ability to put meat on the bones of numeric findings, story-telling methods are useful for addressing many evaluation challenges. Consider using stories to:

• Describe community need: When pursuing funding to address a community issue, the most compelling data you can collect are often the true stories of individuals in need.

• Explain the numbers. If a survey generates hard-to-explain or inconsistent findings, follow up with a few stories to illuminate, validate, or further elucidate the numbers.

• Illustrate individualized results. When numerical outcomes are so diverse that they become neutralized when averaged together, use stories to convey remarkable individualized outcomes that the numbers do not convey.

• Show progress that precedes outcomes. When program outcomes are difficult to achieve (stop smoking; lose weight), capture intermediary shifts in attitude and behaviors through individual stories.

• Capture unexpected or remarkable outcomes. Use stories to explore the range of client successes you could not have imagined when you launched the program.

• Identify patterns and trends. Several stories can be collected on one topic or in response to one broad question and then examined to uncover patterns or trends.

The evaluation story is a great tool for connecting to a program or organization on a personal level.  Everyone loves a good story. The trick as an evaluator is to tell a story that resonates with the audience, and yet is grounded in rigorous qualitative methodology.  Is it a challenge?  It sure is. But the benefits of influencing policy and program development with a well-executed story, make it all worth it.