Hi, I’m a program evaluator

The most popular question at social events seems to be some variation of “what do you do for a living?” In my experience, the topics I’ve focused on thus far in my career can shut down a dinner party conversation faster than you can ask what’s on the menu. Here’s a spark notes version – homelessness, anxiety, sexual violence against women, substance use, child welfare, and juvenile justice. Apparently these are topics that not everyone feels comfortable discussing at happy hour. I’ve also found that people are not that excited to discuss data or statistics. Sound familiar?

Given these reactions, I’ve contemplated developing a professional happy hour-friendly alter ego. But that would be doing a disservice to our wonderful field of evaluation and my quest to champion the implementation of evaluation and data-driven decision-making. Instead, my response usually sounds something like, “I’m a program evaluator, which means that I help organizations understand their data so they can make their programs even more effective.” My email signature is similar and says that “I make data accessible to improve outcomes and communicate impact.”

3At this point the conversation usually goes one of three ways:

  • The person stares at me blankly and the conversation is steered in another direction
  • The other person thinks that I’m an auditor or accountant
  • The person seemingly loves data and nonprofits as much as I do and asks lots of questions

Here’s the thing – People use data and evaluation in their daily lives. Let’s consider Facebook as an example. Facebook users rate the value of their posts by looking at the number of likes, shares, and comments often on a daily, if not more regular, basis. This is evaluation! People also cook recipes and make adjustments to the ingredients when something doesn’t turn out as desired. This is data-driven decision-making!

So this is what I’m wondering:

  • If people use data and evaluation in their everyday lives, why don’t people know more about program evaluation? We just finished the International Year of Evaluation (2015)!
  • How are we talking about and marketing evaluation as a profession?
  • As an evaluator, how do you define evaluation and how do you answer the question, “what do you do for a living?”

12

 

The Power of Pie

If there’s one thing I know for sure after working in different sectors it’s that food brings people together. I mean, have you ever seen your colleagues react as fast to a non-emergency email as when it says ‘extra food in the kitchen, help yourself’? Suddenly the office track team comes out of hiding with cupcakes replacing medals for prizes. Funny, my emails about data meetings don’t seem to incite the same Olympic speed response or enthusiasm.

Then one day I baked a pie for a meeting at which data visualization was the key topic. It was a bit of a jest to highlight that the only 3D pies that are welcome in the reporting process are those that are savory or sweet and can be consumed with a fork. That was the day I realized the power of pie. (Real pie brings people together, pie charts are a highly divisive topic.) Once snacks were provided it seemed that people enjoyed coming to the data meetings. The pie served as an ice breaker with the silver lining being that even with difficult discussions on the agenda, people still get a delicious snack. This effect is not unique to pie, I’ve observed similar reactions to coffee, bagels, and sandwiches. I just really like pie.

The use of food is not limited to an enticement to get people to meetings, it can also be a great way to disseminate evaluation findings. For example, Stephanie Evergreen wrote a blog post, which is absolutely brilliant, about putting results in Findings Cookies. It’s a little data surprise in a delicious cookie – win/win!

  • Looking for something less advanced? Pick up a package of blank toothpick flags, write a finding on each, and stick them into cupcakes.
  • Feeling even more adventurous? Print your findings on edible fondant and place on top of cupcakes or brownies and your results will literally be the icing on the cake!
  • Not a fan of baking? Create your own evaluation chocolate bar wrappers. Avery address labels work well for mini and full-size chocolate bars or you can order blank wrappers from Online Labels.

Findings Chocolate Bar Wrapper

Your treats will not only be delicious, they will also be informative. Plus you have limited space so your text has to be clear and concise. There are so many ways to incorporate food into the data dissemination process, it’s sure to keep your colleagues and partners happy. Leave a comment and share how you’ve practiced the power of pie in your own work. Happy baking!

Crossing the Communication Chasm

Communication and evaluation are everyday activities. We evaluate what the weather is like, how our day went, if our work presentation was effective, and we share that information with others in our life. But when the stakes get higher and we add more data into the mix like when we’re working on an evaluation project, sometimes our communication gets a little wonky.

Sometimes we forget about the many factors to consider like:

  • Who is the audience?
  • What is the purpose?
  • What is the best format (one pager, dashboard, brief)?
  • What is the key message?
  • What language resonates with our stakeholders?
  • Why are we disseminating this information?
  • What technology best meets our needs/does our audience use?
  • Are there any politics (within the organization, community, or nationally) to be aware of around the findings?
  • What resources do we have to dedicate to dissemination?

When thinking about all these factors one point remains central – context is key. Readers will only see what you present. Often this is just the tip of the iceberg so help them understand anything below the surface that might be important.

Don’t assume that you’ve communicated your message well because the information has been disseminated. You may have created a beautiful, well-formatted deliverable. But pretty does not mean effective. Repeat after me: Pretty does not mean effective! To be utilized, the deliverable must meet the needs of the stakeholders. You have to talk in their language, use technology they’re familiar with, and present the findings in a way that speaks to them. Tell me why I should care about the results – paint the picture for me, don’t make me connect the dots. Misrepresentation and miscommunication are likely to occur when people are left to read between the lines.

6

Another way to connect the dots and streamline communication is by engaging stakeholders from the beginning. Think about using this three-step process: listen, design, refine. I recently created a dashboard and having the input of the stakeholders from the beginning to determine the most important metrics and the format was critical. Doing so reduced the number of revisions throughout the process, increased community buy-in, and established a culture of data-driven decision-making from the beginning.

So what does this all mean? Communication is the foundation of evaluation and good communication is imperative for evaluation utilization. So start conversations about disseminating findings early and have them often. And remember to tailor the deliverable to the stakeholder. Good luck and happy reporting!

Happy National (Evaluation) Book Lovers Day

Books have always been a place of refuge for me. They take me on new adventures, challenge my thought process, and expand my knowledge base. There’s something uniquely powerful about cracking open a new book and consuming every word. I love extracting facts, quotes, and snippets of wisdom and adding them to my personal bucket of knowledge. So it comes as no surprise that I chose the profession of evaluation, which affords me opportunities for continuous learning about programs, policies, methods, and research.

To help spread more knowledge and to celebrate National Book Lovers Day (August 9), here are some of my recent and all-time favorites related to evaluation:

Consulting

Data & Data Analysis

Data Visualization

I would love to hear from you! What’s on your evaluation bookshelf?