Adaptive Learning Capacity: Noticing What Your Reports Can’t Show You
This is part 2 of Adaptive Learning Capacity, a five-part series on how organizations learn while the work is happening. New to the series? Start with part 1.
In the first post in the series, I introduced Linden Pathways, a composite workforce program that trains adults for healthcare jobs. After three good quarters, Linden found out almost by accident that the people it was designed to serve, most single parents and people working night shifts, were quietly dropping out, while the people finishing were the ones who needed the least help. The program director learned this because she happened to stop by a case manager’s desk.
What I didn’t mention is that the signs had been there for months.
In the first post, I describe adaptive learning capacity as an organization’s ability to learn while the work is happening, and I broke it into three things people do over and over: noticing what’s changing, making meaning of it together, and responding. This post is about the first of these. Noticing sounds like the simplest of the three, and in some ways it is. It’s also where Linden lost the most time.
Looking back, Denise could trace it to late spring. A few participants started missing Thursday sessions, which fell right when several hospitals and warehouses in town rotated their night schedules. She mentioned it in a team check-in, and someone suggested sending reminder texts. Around the same time, the other case manager noticed that more new referrals were coming from the community college’s career center and fewer from the family shelter that had been one of Linden’s founding partners. She found it interesting and didn’t think much more about it. And at a graduation that summer, a participant told the program director how glad she was to finish, “since most of the moms from my cohort had to drop.” The director remembered that comment later. At the time, it was a graduation and everyone was celebrating.
None of these would have set off an alarm on its own. Each one was small and easy to explain, and each one landed with a different person who already had a full plate. If you’ve ever looked back on a problem and realized the clues were sitting in plain sight, you know how this goes. Nobody at Linden was careless. They were busy doing the work, and the work didn’t come with a place to put observations like these.
We notice what we decide to measure
Linden tracked what its funder asked it to track: enrollment, completion, and job placement. Those are reasonable things to want to know, and the workforce board needed them to justify its own funding. They were chosen when the program was designed, though, based on what everyone expected to matter. Nobody built the reports to show who was completing, because when the program launched, the program enrolling and the people finishing were largely the same people.
This happens almost everywhere. The things we measure reflect what we expected to happen, and when the work unfolds differently, our attention tends to stay fixed on the original plan. Most organizations get quite good at seeing whether they’re on track and have much less practice seeing what’s shifting underneath.
Noticing is something you can get better at
It’s tempting to think of noticing as a personality trait that some people just have. I think if it more as a practice. It starts with curiosity about why something is happening, careful attention to what’s actually going on rather than what you expected, and a habit of listening to the people closest to the work. It also takes a willingness to wonder whether your current explanation is the whole story.
A lot of it comes down to recognizing learning moments, the points where it’s worth stepping out of execution mode and getting curious. Most of your day is execution, and it should be. You can’t stop and analyze every missed session. Some moments deserve a second look, though. A surprise is one, when something goes much better or worse that you expected. A contradiction is another, like a graduate celebrating her own finish while mentioning that most of her cohort didn’t make it. So is a pattern that keeps repeating, like Thursday absences. Sometimes it’s an assumption suddenly becoming visible: Linden had always assumed its referral partners sent the same kinds of participants year after year. And sometimes it’s a shift in context, like a major local employer changing its shift schedule.
Linden had at least four learning moments in about three months. What it didn’t have was a habit of recognizing them.
This is where the difference between the skill and capacity start to matter. Denise, Marisol (the other case manager), and the program director were all good at noticing. Each of them picked up on something real. What Linden lacked was a way for those individual observations to come together into something the organization could see.
Once you do recognize one, a good learning question helps you decide where to look next. For Linden, it might have been something like: What will it take for participants with the most barriers to stay with the program through completion? (I’ve written more about learning questions and how they differ from evaluation questions.) A question like that points your attention somewhere before you start collecting anything new. Often the information turns out to be there already in case notes, attendance records, and staff conversations, waiting for someone to put it together.
Sometimes not noticing protects something
There’s a harder part of this that I want to be honest about, because I think many of you will recognize it.
Linden’s completion numbers were its success story. They were in the grant reports, the board packet, and the annual appeal. The workforce board had just renewed funding for two more years, partly because of those numbers. Staff were proud of them, and they had worked hard for them.
So when small signs suggested those numbers might be hiding something, there was a quiet pull toward explaining the signs away. Nobody decided to look away. It was simply easier to accept explanations that kept the story intact (reminder texts will fix the Thursday problem, referral patterns always go up and down) than ones that raised questions about the program itself. And the people best placed to press harder were the same people carrying the heaviest caseloads.
I don’t see this as a failure of character. When an organization’s funding, reputation, and staff morale are tied to a particular set of results, it makes sense that people are a little slower to take in evidence that complicates those results. It’s human, and it often protects things that really do matter. It’s still worth knowing that it happens, because the signals most likely to slip past you are often the ones that would be most inconvenient to hear.
Whose noticing counts
Organizations don’t see through one set of eyes. The executive director sees one part of the picture, the program director another, and case managers another still, while participants experience things none of the staff ever see. The people closest to a change are usually the first to sense it, and they’re often the people with the least power to make anyone else pay attention.
What Denise had was one observation, from her own caseload, about a handful of participants. In most organizations, that gets filed under “worth keeping an eye on.” (I’ll come back in a later post to how you weigh a single signal like that, because it’s trickier than it sounds.) The graduate who mentioned the other moms in her cohort had no channel at all beyond a passing comment at a celebration.
This is the point where noticing stops being an individual skills and becomes a question of organizational capacity. You can employ wonderfully perceptive people and still have an organization that struggles to notice. If raising a concern tends to get you labeled as negative, people stop raising concerns. If participant feedback gets collected but never makes it into the conversations where decisions happen, the organization is listening without quite hearing. And if leaders mostly ask whether things are on track, people learn to report on whether things are on track.
One of the most useful things a leader can do is ask for the uncomfortable information, out loud and often, especially from people with less positional power. Very often the signal is already sitting with somebody nobody thought to ask.
A note for funders
If you’re reading this from a foundation, part of this story belongs to you. The reports you ask grantees for shapes what they pay attention to. Linden tracked completion because the workforce board asked about completion, and over time that report became the lens the whole organization looked through. That’s nobody’s fault, and funders do need to know whether their investments are working. When you design reporting requirements, though, ask yourself what they make easy for grantees to see and what they might make easy to miss. A single question like What has surprised you this year? can open up a conversation that a table of outcomes rarely will.
What this might look like for you
Your version of Denise’s missed Thursdays might look different. At a foundation, it could be the same question coming up in three separate grantee check-ins. In a coalition, it might be a partner who has stopped sending anyone to meetings. In a small program, it could be the thing your front desk hears every week that never shows up in a report.
Wherever you sit, a few simple practices can help those signals surface sooner. None of them requires a new system.
A regular frontline check-in. Fifteen minutes a month between the people closest to the work and the people making decisions, built around one question: What are you hearing that we might not be?
A piggyback question. Add one open-ended question to something you already use, like an intake form, an exit conversation, or a grantee report: Is there anything that surprised you?
A near-miss log. A shared page where staff can jot down things that almost went wrong. Near misses often point to patterns long before anything actually breaks.
Outside eyes. Once or twice a year, invite someone less attached to your current assumptions, like a board member, a peer organization, or a former participant, to ask what you might be overlooking.
Pick whichever one fits a gap you already suspect. One practice you keep up will teach you more than four you try for a month.
One question to try
You don’t need a new system to start noticing more. At your next team meeting, alongside the usual updates, try asking: What are you noticing that you don’t quite know what to make of yet?
Then resist the urge to fill the silence or solve whatever comes up. Listen for surprises, contradictions, and things you’ve heard before. Maybe nothing will come of it. Or maybe someone will mention something like the Thursday absences, and someone else will say they’ve been seeing it too.
Noticing is only the first move, though. Once Linden finally saw what was happening, the team had to figure out what it meant, and that’s the second part of adaptive learning capacity: making meaning together. They very nearly settled on an explanation that would have sent them in the wrong direction. That’s where we’ll pick up next time.
The Adaptive Learning Capacity series:
Noticing What Your Reports Can’t Show You (you are here)
Getting Past the First Explanation (coming 10/20)
Learning Only Matters If Something Can Change (coming 10/27)
Building the Habits That Make Learning Last (coming 11/3)
