The questions that lead to better decisions
A few months ago, I was facilitating a planning session with a leadership team that wanted to strengthen how they measured their work. The conversation quickly turned to metrics. Someone suggested tracking participant retention, another wanted to measure partnership quality, and someone else proposed adding quarterly surveys. Within 15 minutes, the whiteboard was full of indicators.
Finally, I interrupted: “What are you hoping all of this will help you decide?”
The room got quiet. After a long pause, someone said, “I guess we haven’t talked about that yet.”
We spent the next 30 minutes having a completely different conversation. It wasn’t about surveys or dashboards anymore. It was about a real decision they were facing: whether to expand a successful program into a new community. They weren’t actually wondering if the program was “working.” They were wondering what it would take for it to work somewhere else.
That one shift changed everything. Instead of asking, “What should we measure?,” they started asking, “What do we need to learn before we make this decision?” The metrics came later… and they were much better, because they were finally designed to answer a real question.
The question behind the question
I see this pattern constantly. Organizations skip straight to what should we measure before they’ve clarified what we’re actually trying to learn. When that happens, evaluation quietly turns into an exercise in data collection instead of a practice of learning.
So when I start working with an organization, I ask something deceptively simple: What are you hoping to learn?
The answers are usually well-intentioned - we want to know if our program is working, we want to understand our impact, we want better data. None of those are bad goals. But they’re too broad to actually guide anything.
So I ask a second question: What decision are you trying to make?
That’s when things get concrete. Someone starts talking about whether to expand a program, redesign a service, strengthen a partnership, or invest in a new strategy. Suddenly the uncertainty has a shape and the conversation becomes useful.
Better decisions don’t start with better metrics. They start with better questions.
Two kinds of questions, doing two different jobs
People often use “learning question” and “evaluation question” interchangeably. But really, the play very different roles.
Learning questions are the big strategic questions that help an organization navigate uncertainty. They’re forward-looking, they surface assumptions worth testing, and they focus attention on the decision ahead. They usually start with:
What will it take…
Under what conditions…
How might we…
What are we learning about…
Going back to that leadership team: their learning question turned out to be “What will it take for participants to remain meaningfully engaged throughout the first year in a new community?” That’s not a question you answer once. It’s one that shaped your learning over months, maybe years.
Evaluation questions are narrower. They’re what you actually go investigate to generate evidence that informs the learning question. For that same team, the evaluation questions looked like:
How many and which participants remain engaged after 3, 6, and 12 months?
Why do participants stay or leave?
In what ways do various participant groups engage differently?
What practices lead to greater engagement and retention?
No single evaluation question answers the learning question on its own. But together, they build the picture the team needs to inform their decision about what to do next. That’s the real job of evaluation - not the goal itself, but one way of generating evidence in service of strategic learning.
Don’t stop with the question. Name your assumptions.
Every strategy is built on assumptions about how change happens, whether you’ve said them out loud or not. For the engagement question above, the team likely already believed things like:
Relationships with staff matter more than program content
Participants who connect with peers are more likely to stay
Transportation is a bigger barrier than motivation
Naming these assumptions out loud is what turns evaluation from documenting the past into an opportunity to learn your way forward. If you don’t know what you already believe, you don’t know what you’re actually testing.
A simple sequence to run this yourself
Here’s the sequence I use with teams, start to finish:
Start with strategic uncertainty. What important decision are we facing? What don’t we understand yet?
Develop a learning question. What will it take…? Under what conditions…?
Surface your assumptions. What do we currently believe, and why do we think that’s true?
Develop evaluation questions. What evidence would test those assumptions and inform the learning question?
Make sense of what you learn. What surprised us? What patterns are emerging?
Adapt. What do we continue, change, or ask next?
Without a learning question, evaluation becomes an exercise in collecting information for its own sake. Without evaluation, learning is just intuition dressed up as strategy. The value comes from connecting the two.
Try this before your next meeting
The next time someone on your team asks “What should we measure?,” pause before answering. Write down the actual decision you’re trying to make. Then ask: What would we need to learn to make that decision well?
That one substitution - trading “what should we measure” for “what are we trying to learn” - is a habit that changes how an entire organization thinks about evidence. Because better decisions don’t start with better metrics. They start with better questions.
