For the last few Octobers, my town has turned into Halloween central. I’m talking over the top decorations. It’s been a strange juxtaposition, watching the goofy yard displays pop up alongside the rise of fascism. I’ve found it increasing difficult to focus on silly things when equity, justice, and our field of evaluation are actively being challenged and dismantled. But perhaps that’s exactly what we need right now — to find the small joys and creative outlets in our communities.

I know I’ve certainly needed more levity in life. So when the giant 12-foot skeletons started appearing a few years ago, I decided to have some fun with it. I began keeping a running tally of the giant decorations I passed in town. Friends started to join the adventure, sending me tips on places to look. (I’m not here to put photos of other people’s houses on my website, so you’ll have to use your imagination)

When Count Skelly started, this was a pretty simple task — count the 12-foot skeletons. Easy-peasy…right?

Wrong.

As the years have gone by, the options for giant skeleton decorations have expanded and diversified, challenging the count criteria. For example:

• Do I count the 5-foot Giant Ground Breaking Skeletons?
• What about the 5-foot Giant Pumpkin Head Ground Breakers?
• How about 12-foot skeletons with pumpkin heads?
• What about 12-foot scarecrow skeletons?
• Do the 12-foot werewolves count?

I mean, they’re all still technically giant skeletons. They literally have “giant” in their product name. So it seems like they would count.

And now the skeletons have pets, raising an entirely new set of questions:

• Do the 5-foot dog and cat skeletons count?
• How about 6-foot rooster skeletons?
• Where do horse skeletons fall in the count?

But if I include the 5 and 6-foot animal skeletons, do I need to also include the 8-foot regular skeletons?

Oy vey. For a seemingly simple situation, this count has gotten increasingly complicated.

And that, my friends, describes every evaluation project ever. What initially seems like a simple data question is often steeped in nuance and context, requiring thoughtful discussion, care, and consideration.

Now in the comical world of counting Halloween decorations, the rules are made up and the potential for negative ramifications is non-existent. So my data definitions about what is considered a giant skeleton can shift and change based on my discretion.

But during a real evaluation project, the stakes are much higher.

The definitions and criteria we use to include or exclude information can be the difference between whether a family receives services or a program is granted funding. It is our responsibility, with our partners, to establish clear, consistent, accurate, and transparent boundaries around our data definitions to promote:

• Alignment with program goals: Data definitions that connect back to the underlying program theory and implementation ensure what is being measured reflects the reality.
• Data quality and integrity: Documented data definitions help to improve consistency across staff, ensure replicability over time, and reduce bias and errors.
• Effective decision-making: Reliable data supports leaders in making informed judgments about program effectiveness and improvements.

Articulating these boundaries is one of the steps I take with clients when we are exploring what information they already collect and what they may want to gather in the future to tell their story. During this discovery process, we typically create a data inventory (the who, what, where, when, and why of each data source) and develop a data dictionary that documents how each data element is being collected.

Because we love a rebranding in evaluation (sigh), you will likely see similar tools online under slightly different names. I’ve found the Virginia Early Childhood Foundation’s ​Data Inventory Template​ and We All Count’s Data Biography Template​ to be wonderful resources.

Have you used a data discovery and definition process? Reach out and let me know the biggest a-ha you learned.

Or think this process might be helpful for your team in the new year, let’s chat!

Ps – If you’re curious, the 2025 count is up to 72 giant skeletons. The current count criteria include any skeleton that is significantly larger than reality: all 12-foot skeletons, 6-foot roosters, and 5-foot dogs and cats. But not horses, because horses are already large IRL. Super scientific, I know.