Monitoring & Evaluating Capacity Strengthening: How Do We Measure Long-Term Impact?
@all Most capacity strengthening projects measure short-term outputs such as the number of workshops held or researchers trained rather than long-term systemic impact, such as institutional autonomy or policy reform.
What frameworks or indicators effectively capture long-term, institutional capacity growth over 5–10 years?
How can funders and institutions align on meaningful M&E metrics beyond quick post-workshop evaluation forms?
Thank you, Nonvicks, for raising this important issue. From a health systems, monitoring and evaluation, and implementation research perspective, I believe that measuring long-term capacity strengthening requires a shift from counting activities to assessing sustained institutional performance, autonomy, and influence.
1. Frameworks and indicators for measuring long-term institutional capacity
A combination of frameworks is more appropriate than relying on a single model:
-
Outcome Mapping and Outcome Harvesting: Track behavioural and institutional changes, including how researchers, research institutions, policymakers, and implementing partners use evidence over time.
-
Contribution Analysis and Theory of Change: Assess how capacity-strengthening investments contribute to institutional outcomes while accounting for other contextual influences.
-
Organisational capacity assessment frameworks: Measure changes in leadership and governance, research management, financial sustainability, workforce retention, data systems, grant management, and internal quality assurance.
-
Research capacity strengthening frameworks: Assess the ability of institutions to independently formulate research agendas, secure competitive funding, conduct high-quality research, retain talent, and translate findings into policy and practice.
Over a 5–10-year period, useful indicators include the proportion of research funding managed independently by local institutions; retention and career progression of locally trained researchers; institutional overhead and indirect-cost recovery; locally led publications and grants; functioning research ethics and data governance systems; sustained research partnerships; and documented use of evidence in policies, guidelines, and service delivery.
2. Aligning funders and institutions around meaningful M&E metrics
Funders and recipient institutions should jointly establish a Theory of Change, baseline institutional capacity assessment, and a small set of agreed long-term outcome indicators before implementation. These should be complemented by annual organisational assessments, independent evaluations at years 3–5, and longer-term reviews at years 5–10.
Importantly, indicators should distinguish between outputs, intermediate outcomes, and systemic impact. Workshop attendance and post-training satisfaction are useful implementation measures, but they do not establish whether institutional capacity has improved or endured.
A balanced measurement framework should combine quantitative indicators with qualitative evidence, institutional self-assessment, external verification, and stakeholder perspectives. It should also account for context, equity, staff turnover, and the time required for institutional reforms to mature.
Ultimately, the measure of successful capacity strengthening is not simply how many people were trained, but whether institutions can independently sustain, finance, lead, and apply their capabilities long after external support ends. Funders should therefore invest in long-term measurement, adaptive learning, and institutional ownership rather than short funding cycles and activity-based reporting.
@Dr. Magoba Thank you, Dr. Magoba, for this comprehensive breakdown. The distinction between counting activities and measuring sustained performance especially through indicators like independent grant management and talent retention really hits the nail on the head.
Building on your points regarding funder alignment:
-
In practice, long-term impact evaluations (5–10 years) often face the challenge of short funding cycles (2–3 years). How can institutions structure grant proposals to incentivize funders to invest in longitudinal M&E or post-project evaluations?
-
With frameworks like Outcome Harvesting, what strategies work best to manage the resource-intensive nature of qualitative data collection across multiple regional contexts?
@Rhoda Nakhosi This is a super clear, practical framework for operationalizing long-term M&E! Tiering indicators and shifting funders toward milestone reporting (years 3, 5, 10) makes so much sense for tracking genuine autonomy over quick output metrics.
To your question on Organisational Capacity Assessment (OCA) tools that adapt well across contexts:
-
USAID’s Organisational Capacity Assessment (OCA) / Organizational Performance Index (OPI): Highly adaptable and focuses on both internal governance systems and performance outputs.
-
McKinsey Capacity Assessment Grid (OAT): Great for evaluating leadership, operational management, and financial sustainability.
-
IDRC’s Institutional Assessment Framework: Tailored specifically for research institutions in LMICs, focusing on research governance, infrastructure, and policy influence.
The biggest challenge often lies in keeping self-assessments objective. In your experience, what has worked best to ensure institutional self-assessments stay honest and actionable rather than just becoming a exercise to satisfy funders?