Every coach and scholar in our program gets asked the same question once a month.
How are things going?
It’s a simple question and for years, getting a real answer was some of the most expensive work we did.
Nobody likes surveys.
When we started, we did what every organization does: Send a monthly survey.
Let’s be honest about surveys. Nobody wants to click a link and spend five minutes in a Google Form.
So people didn’t fill them out, and the ones who did were usually the ones doing fine. The relationships we most needed to hear about were the quietest.
So, we scrapped the survey and went personal - texts, phone calls, real conversations. That was better, but it was a grind. You catch someone’s voicemail, send a follow-up email, you don’t hear back, you make a note to circle back next week. Do that for every coach and scholar in the program and you’re chasing more than 60 people, every month.
And even when the conversations happened, every one was different. One coach tells you all about their mentees internship. Another mentions they haven’t connected in a while. You hang up feeling informed, but nothing you learned lines up with anything else you’ve collected.
Then every conversation had to get logged somewhere. On a good day, that was Salesforce. Most days, it was somebody’s memory - and memory is a terrible database.
The tax nobody counts.
There’s perhaps a bigger cost that never shows up on a time sheet.
Our Director of Student Success and I would spend part of every week just reconstructing the state of the program. Did you talk to her this month? Did he ever get back to you? When’s the last time anyone heard from that pair? We weren’t solving problems in those conversations. We were figuring out what we knew.
And the chasing didn’t stop at check-ins. It’s was the same challenge everywhere. Are you coming to the finance dinner? Did you RSVP? Can you commit to Match Day? Different question, same loop: reach out, wait, follow up, log it, repeat.
I never added it up until I sat down to write this. Between the check-ins, the event follow-ups, and our weekly back-and-forth about who’s doing well and who needs help, this work was eating close to half of his working hours. Half of a role, spent asking questions we never got consistent answers to.
The answer to “how are our relationships doing” was a feeling. A well-informed feeling, most of the time. But still, just a feeling.
Four taps.
We built a mobile app our coaches and scholars already use, so the solution was right in front of us.
At the start of every month, the app asks each person four questions about their mentoring relationship. Have you connected in the last 30 days? Do you feel supported? Rate the relationship, one to five stars. Is there anything Level Up can help you with?
Four taps. Under 30 seconds. Done.
For the first week it’s a gentle banner. After that, the app politely insists - you check in before you go on with whatever you opened it to do.
If somebody says they need something, our team gets an email within seconds, not next month when we happen to call. And every response lands in a permanent history for that relationship, month over month, ready to sync into our CRM. The same questions, asked the same way, answered by everyone. For the first time, the answers can actually be compared.
Here’s the metric I didn’t see coming: silence. If someone doesn’t check in, it means they’re not opening the app at all. That’s not missing data. That is the data.
I built the whole thing in a day - the check-in flow, the reminders, the staff alerts, the dashboard that shows every relationship’s trend line. One working session, and only because everything underneath it already existed: the app was built, the admin dashboard was live, the notification plumbing was wired in. This feature was a cherry on top of a cake we’d already baked. I wrote about that in We Were the 7% Before AI Got Here.
The part I care about most.
What actually changed has nothing to do with data.
When our Director of Student Success reaches out now, he’s not gathering information. That’s done - he’s just building a relationship.
The call used to open with a checklist. How often are you meeting? How’s it going? Anything you need? Now it opens like a conversation with someone you know. How are you? How’s the family? I saw your check-in, sounds like things are going great.
We’re more present with people because we’re not interrogating them.
His time goes where it matters, too. The dashboard surfaces the relationships that are struggling, so he starts there instead of working down a list. A thriving pairs get a lighter touch. The quiet ones get found.
Launch week humbled us.
We flipped it on August 1. By day four: zero check-ins.
Zero. I kept opening the dashboard thinking the number had to be wrong.
The reminder notifications were sending, or so I thought. When I dug in, I found something I didn’t expect: our iOS push notifications had never been configured for production. Not for the pulse - for anything. Every notification we’d ever sent to an App Store user had silently failed, and our tests always passed because tests run in a sandbox.
One missing upload in a Firebase console. It had been that way since the day we launched the app.
I fixed it that afternoon and re-ran the send. 100% delivery.
We thought notifications were a solved problem. They’d been “working” for a year. It took building something new on top of them to find out they never were.
That’s the thing I love about continuous improvement. It isn’t only that you keep making things better - the act of building keeps surfacing things you didn’t know were broken. Every new layer puts weight on the layers underneath, and weight finds the cracks!
The rollout wasn’t clean but I’m glad it wasn’t. A clean rollout would have left that issue buried.
One Month In.
24 of the 32 check-ins we expected came in. 75%, in month one.
The average relationship rating was 4.7 out of 5, and 96% of people said they feel supported. Two people told us they needed something: a scholar who’ll need help with textbook expenses when fall semester starts, and a coach who was having trouble reaching her mentee. We knew about both within seconds of them tapping submit, and both were being worked before the month ended.
Under the old system, that student need surfaces in October (maybe). The coach quietly keeps struggling until someone happens to ask.
The follow-up with the eight who didn’t check in took minutes, not weeks, and it sounded completely different. Not “hey, did you ever get to that survey I sent you?” Just “did you hear about the new monthly pulse in the app?”
I expect these numbers to improve from here. Habits take a few cycles, and the app will ask again next month, and the month after, exactly the same way.
Where the hours go now.
The hours we’re getting back don’t disappear. They move.
They go to planning the events where coaches and scholars actually build connection. To coach training. To reach-outs that are personal because they can be. The same plumbing now nudges people about events too, so the RSVP chase is fading the same way.
Put a conservative price on all that chasing and it’s over $15,000 a year in staff capacity. Every hour of overhead we automate is an hour that goes back to students.
We still ask the same question every month. How are things going?
The app asks it now. Which frees us up to ask a better one.
How are you?
The toolbox.
Here’s what I used to build the monthly pulse:
- Claude Code: built the entire feature in one session
- Firebase: stores the check-ins, sends the reminders, triggers the alerts
- Expo / React Native: the mobile app it lives in
- Postmark: delivers the staff alert emails
All free or close to it.
---
The $0 Department: What Happens When a Nonprofit Stops Hiring and Starts Building. This is what it looks like when there’s a story worth telling. Subscribe to catch the next one.
Level Up Cincinnati empowers first-generation college students through mentorship, scholarships, and professional development. 100% of donations go directly to student programming. Learn more at levelupcincinnati.org