Wearing Many Hats: Where Does the Role of Evaluator End?

Four years ago I uprooted my life and moved halfway across the country to relocate to a state that had never been on my radar. In that time, I’ve established myself in the evaluation community, working both as an external evaluator and as a consultant building the evaluation capacity of organizations. You would think in that amount of time, plus my years of experience prior to this, that I would have a clear answer about my professional identity and how it fits into the broader field. But really, I feel that I find more questions than answers with each passing day.

If you’ve ever tried to explain program evaluation to a non-evaluator, you know what I’m talking about – the glazed over looks you get when mentioning that you’re an evaluator, the people who think you’re a researcher or auditor, and the others who assume you’re a computer programmer. I also received my MSW so I’m fairly confident that part of my family still thinks I’m a therapist, despite the fact that my degree is in policy and evaluation. Add to that the many hats that a program evaluator wears in the 21st century, and it’s really a wonder that we haven’t created a support group. Or maybe we have – shout out to my #EvalTwitter community!

Evaluators wear many different hats, which vary depending on the evaluation approach as well as the stage of the evaluation cycle. Over the years, evaluation theorists have debated the role of the evaluator. Campbell classifies the role of evaluator as methodologist, Scriven says judge, Stake says facilitator, and Wholey says educator (Luo, 2010). Today, evaluators often straddle all of these roles in addition to new roles brought with advancing technology and globalization such as designer and marketer. Other hats include data analyst, project manager, grant writer, strategic planner, coordinator, educator/teacher, coach, and facilitator.

While it is clear that evaluators operate under the American Evaluation Association’s Guiding Principles: systematic inquiry, competence, integrity, respect for people, and common good and equity, the role(s) of an evaluator are not as clearly demarcated. In fact, a commonly cited quote in the field of evaluation is: “evaluation – more than any science – is what people say it is; and people currently are saying it is many different things” (Glass & Ellett, 1980). Even the American Evaluation Association (2004) states that, “based on differences in training, experience, and work settings, the profession of evaluation encompasses diverse perceptions about the primary purpose of evaluation.” If the evaluation field struggles to uniformly define the scope of evaluation roles, it is likely that stakeholders and clients are unclear about what services evaluators offer.  

Two years ago, I led a Birds of a Feather discussion at the American Evaluation Association conference seeking clarity about the many hats an evaluator wears. I was curious to understand:

  • What are the hats evaluators wear?
  • Which of these hats, if any, are in conflict with one another?
  • What are the boundaries of evaluation?
  • How do we communicate these boundaries to our clients?
  • How might globalization change the role(s) of an evaluator?
  • To what degree can and should evaluators play a role in policy change, advocacy, and program development?

Attendees shared that they feel that evaluators can bridge all of the aforementioned roles as long as we acknowledge our biases at the outset of a project and are transparent about our work. The group unanimously shared that open communication is the most important aspect of an evaluation partnership, more so than the specific role(s) being occupied by the evaluator. Attendees also suggested that evaluators develop subjectivity statements at the outset of a project, which according to Given (2008), “is a summary of who researchers [evaluators] are in relation to what and whom they are studying. Researchers [evaluators] develop these from their personal histories, their cultural worldviews, and their professional experiences.”

This all about made my head explode. Have I been overthinking the role(s) of an evaluator and whether and how I fit into the field for no reason? Have I been questioning the ethics of whether we can occupy all of these spaces and still fall under one profession to no avail?

For four years I’ve been pondering – Who am I as an evaluator? Am I even an evaluator? Or am I a social worker with an evaluation and data-driven mindset? Can I be both? Am I a policy analyst who implements data-driven approaches? Am I an advocate that uses data to drive change? How many hats can I wear before my professional identity is so dispersed that it is nonexistent? Does claiming a professional identity even matter?

It seems that, as of yet, there is no consensus about the hats evaluators wear and the boundaries of evaluation. With each colleague I speak with I get another definition or answer, which raises more questions. This reminds me of a quote by Elie Wiesel, “I define myself more by my questions than by my answers, because answers come and go but questions remain.” So for now I will take the attendeesadvice and focus on communication and transparency as I continue to ponder all of these questions.

Interested in pondering these questions with me or being featured in a blog series on this topic? Share a comment or send me a tweet.

Leave a Reply

Fill in your details below or click an icon to log in:

WordPress.com Logo

You are commenting using your WordPress.com account. Log Out /  Change )

Google+ photo

You are commenting using your Google+ account. Log Out /  Change )

Twitter picture

You are commenting using your Twitter account. Log Out /  Change )

Facebook photo

You are commenting using your Facebook account. Log Out /  Change )

Connecting to %s