Long before we entered the pre-Covid/during-Covid realm, marked by daily monitoring of case counts, testing, hospitalizations, and death tolls, I had my own pre/during world. Twenty-two years ago the world as I knew it changed as I was rushed to the emergency room and monitored for swelling in my brain. The doctors weren’t sure I would live through the night. With the uttering of one phrase, one diagnosis—Type 1 Diabetes (T1D), my life changed.

Since being diagnosed with T1D, I have had many lessons, successes, and struggles. My daily routine consists of constant monitoring of my blood sugar levels and adjusting my insulin and food intake accordingly. I feel like I should be awarded an honorary degree in predictive analytics based on the calculations needed to anticipate and course-correct every move.

You would think, being an evaluator, that I relish and thrive in this world of numbers, monitoring, and learning. I mean, I have an insulin to carb ratio so I just need to count my carbs and dose my insulin accordingly, and it’s smooth sailing. Right? Wrong. So many things impact blood glucose levels including fiber, glycemic index, hormones, stress, and type and duration of exercise. Often I just feel frustrated and exhausted. How am I, someone who monitors numbers for a living, unable to crack the secret code?

And why, when I have so many facets to myself, do I constantly feel like my success, value, and worth as a human, as someone with T1D, is being judged by one number—my HbA1c? (HbA1c captures a person’s average blood glucose over the past 3 months and is a key indicator for measuring “control” of T1D) 

So I started to ask myself, if I—someone who loves numbers, data, and evaluation—feel this way, how must my evaluation clients feel? 

Here’s what I know:

Numbers are scary. 

When I was a child I had a physician threaten to confiscate my insulin pump if I did not achieve their desired HbA1c level. To this day, over 20 years later, I am still nervous when I go to the doctor. 

Our evaluation clients are nervous too. They fear that diving into the data could result in punitive recourse in terms of loss of funding or programming. They fear that one number could result in judgement at best or worse – closure. Our job as evaluators is to shift the message from evaluation as punitive to evaluation as learning. We need a stronger focus on curiosity, capacity building, and co-creating rather than indicators and benchmarks.

What you ask matters. 

Nearly every visit to my endocrinologist goes like this: doctor downloads my records into their computer, prints them out, reviews them, and then comes into the room. The doctor then proceeds to ask me what I was doing two Thursdays ago at 3pm when my blood glucose level was high. Now I don’t know about you but I usually have a hard enough time telling you my schedule from yesterday, let alone weeks ago. And yet, they ask. Every. Single. Time. I usually make something up.

We see similar approaches in evaluation. Poor survey and interview designs ask people to recall very specific information across unreasonable time intervals. Then we treat those results as fact. Understanding how to ask the right questions is key to gathering good data. See Sheila B. Robinson and Kimberly Firth Leonard’s book, Designing Quality Survey Questions, for guidance.

Data represents people. 

Another fun exercise endocrinologists and diabetes educators like to do is having you write down everything you ate for the past two weeks. I don’t have time for that. I’m sure it’s very informative but I’m not going to prioritize that into my schedule at the moment. It’s just not going to happen.

Here’s the thing. In evaluation, numbers represent people. People have rich and complex lives that often do not fit into a nice neat checkbox or pass a fidelity check. So maybe it’s time we stopped asking them to. We need to fund qualitative, participatory, and developmental evaluation projects that not only measure but value and align with the lived experiences of program staff, clients, and community members. And we need to stop asking people to show up without compensation for their time, feedback, and expertise.

Context is everything. 

As I mentioned earlier, pretty much anything and everything influences blood glucose levels. I can eat the same thing at the same time and take the same amount of insulin three days in a row and get different results every time. This is because my context is changing every day. Perhaps I went for a hard hike one day and gave a presentation another. Maybe it’s performance review time—definitely need extra insulin for those meetings!

Many qualitative and participatory evaluation approaches recognize context as important contributing factors. In public health, we often refer to Bronfenbrenner’s ecological systems theory. And while we say context matters, we rarely intentionally and thoroughly explore the historical and political roots of whatever social issues a program or project is addressing. We must take a more equitable approach to evaluation and understand these underlying systems. See the Equitable Evaluation Initiative’s framework for inspiration. 

We’re going to mess up.

One thing I’ve learned from having T1D, is that there’s no one size fits all model. I have all of the technology and fancy gadgets and I still struggle.

We’re going to struggle during evaluation too. Each client and project is unique, requiring tailored methods and approaches. What worked with one team may not work well with another. While this can seem daunting, it’s also an opportunity to learn, grow, and challenge our assumptions. 

Recently a colleague told me it’s ok if the project isn’t perfect because there will always be some margin of error. He said this to calm my perfectionism. But I wasn’t worried about getting it right to be perfect. I was worried about the implications. I was worried about the people. About what could happen to the clients or the community if my numbers were wrong. I think about this with my health too—what could happen if my numbers are wrong. And that, my friends, feels daunting. That is why data has power. And that is why I strive to be both gentle and vigilant in my health and my evaluation work.