What Is a KPI?
KPI stands for Key Performance Indicator. It’s a measurable value that reflects how effectively a project, organization, or system is achieving its strategic objectives.
In the context of the Early Signal Project, KPIs help answer questions like:
- Is the system accurately identifying students who need help?
- Are counselors responding quickly enough?
- Are students improving after interventions?
Each KPI is a quantitative signal that turns complex human-centered work into trackable, actionable insight.
Why Defining KPIs Is Important
1. Clarity of Purpose
KPIs force us to define what success looks like. Instead of vague goals like “help students,” we get specific:
“We want ≥70% of flagged students to be confirmed at risk by counselors.”
This clarity helps align our team and stakeholders around a shared vision.
2. Accountability & Transparency
KPIs create a framework for accountability. KPIs show whether our system is working, and where it’s falling short.
- If our precision is low, maybe the model needs better features.
- If response time is high, maybe counselors need more support.
This transparency builds trust with schools, parents, and partners.
3. Continuous Improvement
KPIs aren’t just scorecards, they’re feedback loops.
- We can iterate our model based on precision/recall.
- We can refine our dashboard based on counselor satisfaction.
- We can adjust outreach strategies based on intervention rates.
KPIs help us evolve from “good intentions” to evidence-based impact.
4. Resource Allocation
In nonprofits, resources are limited. KPIs help us prioritize:
- Which schools need more support?
- Which students benefit most from early intervention?
- Which features of our system are worth scaling?
KPIs turn data into decisions.
5. Storytelling & Advocacy
Numbers tell stories. When we say:
“80% of flagged students received support within 48 hours, and 65% showed improved attendance,”
we’re not just reporting, we’re persuading. Funders, school boards, and community leaders respond to evidence-backed narratives.
Good KPIs Are…
| Trait | Description |
|---|---|
| Specific | Focused on one clear outcome or behavior. |
| Measurable | Quantifiable with available data. |
| Actionable | Leads to decisions or changes. |
| Relevant | Tied directly to our mission. |
| Time-bound | Measured over a defined period (e.g., weekly, quarterly). |
Summary
Defining KPIs is like setting the compass for the team. They help us:
- Know where we’re going
- Measure how far we’ve come
- Course-correct when needed
- Prove our impact to others
What does success look like?
- Counselors receive timely, actionable alerts.
- Students flagged as high-risk receive support.
- Fewer students fall through the cracks academically or emotionally.
Our KPIs
| KPI | Description | Target |
| Precision of Risk Flags | % of flagged students who counselors agree were truly at risk. Measures the accuracy of the system’s alerts by calculating the percentage of flagged students who were confirmed by counselors to be genuinely at risk. High precision indicates fewer false positives. | ≥ 70% |
| Recall of Risk Flags | % of truly at-risk students who were successfully flagged. Assesses the system’s ability to identify all students who are truly at risk. It’s the percentage of actual high-risk students who were successfully flagged. High recall means fewer missed cases. | ≥ 60% |
| Counselor Response Time | Avg. time between alert and first intervention. Tracks the average time between when a student is flagged and when a counselor initiates contact or intervention. Shorter response times reflect operational efficiency and system impact. | ≤ 48 hours |
| Intervention Rate | % of flagged students who receive support within 1 week. Indicates the proportion of flagged students who receive some form of support (e.g., counseling session, referral, outreach) within a defined time window (e.g., one week). Reflects follow-through and system integration. | ≥ 80% |
| Student Outcome Shift | Change in GPA, attendance, or sentiment after intervention. Evaluates the change in key student metrics—such as GPA, attendance, or sentiment—after receiving an intervention. Helps quantify the downstream impact of early detection and support. | +5–10% improvement |
| Counselor Satisfaction | Average rating of dashboard usefulness (1–5 scale). Captures counselors’ subjective assessment of the system’s usefulness, clarity, and relevance via a standardized survey. A proxy for usability and adoption likelihood. | ≥ 4.0 |
How we calculate
| KPI | Data Sources | Calculation Logic | Notes |
| Precision of Risk Flags | • Weekly flagged student list (from model) • Counselor validation logs (e.g., “confirmed risk” checkbox) |  | Requires counselors to review each flagged student and confirm whether the alert was valid. |
| Recall of Risk Flags | • Counselor-identified at-risk students (manual list) • Weekly flagged student list |  | Counselors must also log students they believe were at risk but not flagged—this is our “missed” group. |
| Counselor Response Time | • Timestamp of alert generation • Timestamp of first counselor action (meeting, outreach, note) |  | Can be tracked via dashboard logs or a simple Google Sheet with date/time entries. |
| Intervention Rate | • Weekly flagged student list • Intervention logs (e.g., meeting notes, referrals) |  | Define what counts as “support” (e.g., check-in, referral, group session). |
| Student Outcome Shift | • GPA before/after intervention • Attendance before/after • Sentiment scores before/after |  | Use a 4-week window before and after intervention. Can be averaged across students or tracked individually. |
| Counselor Satisfaction | • Post-pilot survey (Likert scale: 1–5) • Optional open-text feedback |  | Include questions on clarity, usefulness, and actionability. Consider follow-up interviews for deeper insight. |
Early Signal Project 