Measuring Success — M&E Framework (Demonstration)

The results chain — instrumented at every link

Every link has a "how we know" — nothing in the chain is asserted without a data source.

InputsVerified modules, LMS, analytics, AI production pipelineAssets produced & verified

ActivitiesTutors and learners use modules; QA tools digitized; staff trainedUsage logs, training records

OutputsDemonstration library + QA modules deployed; staff certified on the systemsDeployment records, completion & certificates

Outcomes — shortKnowledge & correct-technique gains; tutors teach with mediaPre/post assessments, OSCE pass rates, tutor-use survey

Outcomes — mediumCompetent skilled-birth-attendant practice at scaleOSCE at scale, structured observation

ImpactSafer maternal & neonatal careEmONC / MCH indicators — reported as contribution, never sole attribution

Sample indicator matrix

A deliberately small, defensible set — each with a level, source and cadence. Baselines and targets are set with each institution during discovery; leading indicators report within weeks, lagging within months.

Indicator	Level	Source	Cadence	Type
% of target cadre enrolled (Reach)	Institution	LMS cohorts	Monthly	Leading
Active learners per month	Institution	Analytics	Monthly	Leading
Module completion rate	Both	Completion + analytics	Monthly	Leading
OSCE pass rate at the 80% gate	Learner	Quiz reports	Per cohort	Leading
Pre/post knowledge gain	Learner	Paired surveys	Per cohort	Leading
Step-level failure rate (e.g. counter-traction omitted)	Learner	Tagged questions / xAPI	Continuous	Leading
Tutors teaching with the media (self-report + spot check)	Institution	Quarterly survey	Quarterly	Lagging
Observed competence on critical steps	Learner	Observation checklist	Review points	Lagging
Institutional EmONC / MCH indicators	System	Facility data	Annual	Lagging

Anchored to two established frameworks: Kirkpatrick (reaction → learning → behaviour → results) per learner, and RE-AIM (reach, effectiveness, adoption, implementation, maintenance) per institution.

Two lenses on success

The consultant’s scorecard — how I’d be judged

Delivery: every ToR deliverable accepted, on time, against pre-agreed acceptance criteria set in discovery — not negotiated at the end.
Quality gates: 100% of clinical media clinician-verified before deployment; zero unresolved clinical-accuracy findings at handover.
Capacity actually transferred: designated staff in each institution perform the core admin, authoring and analytics tasks unaided, against a competency checklist — observed, not assumed.
Independence after exit: the systems keep running and content keeps being updated 90 days after the mission ends, without the consultant.

The project’s scorecard — how FEXTE can measure Component 1

Outputs: videos deployed and functional per board; staff trained and certified; QA tools digitized and in use.
Outcomes: completion and pass rates against baseline; measured knowledge gain; tutors demonstrably teaching with the media.
Adoption & maintenance: each board operating its own dashboard and issuing its own reports by end of mission.
Impact contribution: movement on EmONC/MCH indicators, reported honestly as contribution alongside the other Component investments.

Baseline first — and a learning loop, not just reporting

Baseline is a Stage-0 deliverable per board: pre-test knowledge, current tutor practice, and existing completion data are captured before rollout — without a baseline, change cannot be proven.

The learning loop: step-level failure data shows exactly which critical step learners most often get wrong (for example, counter-traction omitted during controlled cord traction). Those modules are flagged, revised, and re-measured. M&E feeds revision — the same measure → find the weak step → improve → re-measure logic that governs the authoring pipeline.

Reporting cadence: automated monthly completion + at-risk digest to each registrar; quarterly programme review against the indicator matrix; per-institution dashboards (each board sees only its own cohort) with an EF-level roll-up. Governance: learner privacy, per-institution data ownership, aggregate/de-identified external reporting, and portable records (xAPI) so a board’s data moves with it at handover.