When the Evidence Isn't Ready Yet: How Should Practitioners Make Decisions in the Meantime?
Most discussions of evidence-informed decision making assume a sequence:
generate rigorous evidence, then let it inform decisions. In practice, for many
of us working directly with communities, the sequence is inverted. Decisions
cannot wait for evidence to mature, people are in front of us now, and the
evidence base we would ideally want is still years away.
At Purpose Rwanda, we have built a peer-led addiction recovery model, the
Purposeful Agent of Transformation (PAT) approach, that has produced a 70%
sustained recovery rate across 1,700+ beneficiaries in 5 districts since 2017.
That figure comes from programme monitoring data, not a randomised controlled
trial. We are now pursuing a more rigorous RCT design specifically to test what
we believe we already know from years of direct practice. In the meantime, we
have continued making real-time decisions, who to enrol, how to structure peer
mentorship, when to refer upward to clinical care, using evidence that would
not satisfy a strict EIDM standard, because the alternative was to make no
decision at all while people remained in crisis.
This raises a genuine question for this community: what is the appropriate
epistemic standard for decisions that cannot be deferred? Is programme
monitoring data, triangulated with lived experience and practitioner judgement,
a legitimate basis for interim decision-making while more rigorous evidence is
pursued in parallel? Or does treating it as sufficient risk normalising weaker
evidence simply because urgency is real?
I suspect most of us in frontline or applied research contexts have quietly
developed our own working answer to this tension, even where our institutions
have not formally named it. I would value hearing how others here have
navigated the gap between the evidence we have and the evidence we wish we had,
particularly in contexts where waiting has a real human cost.
Individually weaker, collectively stronger.
Charles, this is an important and very practical reflection. I strongly agree that in frontline public health, evidence-informed decision-making is rarely as linear as “generate evidence first, then make the decision.” When people are already experiencing harm, waiting for perfect evidence can itself become a decision—with potentially serious consequences.
I would argue that programme monitoring data, lived experience and practitioner judgement can legitimately support interim decisions, provided we are transparent about the level and limitations of the evidence. The key distinction, in my view, is between using imperfect evidence responsibly and presenting imperfect evidence as definitive proof.
For me, the appropriate approach is proportionality: the urgency and potential consequences of the decision should determine how much uncertainty we are willing to tolerate. Where the human cost of waiting is high, the best available evidence can guide action, while stronger evaluation—including your planned RCT—continues in parallel. Monitoring data can help us understand reach, retention, implementation fidelity, emerging outcomes and unintended effects, while lived experience helps explain what the numbers alone cannot.
The important safeguard is to build learning into implementation: act, monitor, triangulate, reassess and adapt. In that sense, “evidence-informed” should not necessarily mean “evidence-perfect.” It should mean making the most defensible decision possible with the evidence available, being explicit about uncertainty, and remaining willing to change course as better evidence emerges.
Your PAT experience illustrates this tension beautifully: practice generates learning, learning informs adaptation, and rigorous research can subsequently test whether those observed effects are causal and generalisable. Perhaps the real challenge is not choosing between practice and rigorous evidence, but creating systems where both can continuously inform each other.
Individually weaker, collectively stronger.
this context, particularly because it provides a framework that can
accommodate the demands of a rapidly evolving crisis situation. Where a
crisis requires real-time solutions and immediate decision-making, there
should be room for a rapid assessment that can generate timely and
relevant information for action. In such circumstances, relying solely
on routine monitoring data may not be sufficient, as monitoring systems
are often designed to capture trends over time rather than provide
immediate insight into an emerging situation. It may therefore be more
appropriate to prioritize fresh, compelling, and context-specific
evidence that reflects the current realities of the crisis. A quick
assessment can help gather this evidence within a short timeframe,
identify the most urgent needs and gaps, and provide decision-makers
with the information required to respond effectively. This approach can
ensure that interventions are based on current realities while still
maintaining sufficient rigor to support sound and timely decisions.
I think this is an important distinction, and I particularly agree with your question: “proportional for how long?” Proportionality should not become a permanent exemption from stronger evidence.
My framework is useful because it recognizes that evidence requirements should reflect urgency, risk, feasibility, and the consequences of delay. However, proportionality must also be dynamic. As a programme matures, monitoring systems improve, implementation experience accumulates, and resources become available, the justification for relying primarily on lower-certainty evidence should progressively weaken. Otherwise, an emergency standard can become institutionalized as the default.
I would therefore add a third dimension to the framework: evidence maturity. The question should not only be “What evidence is proportionate to the situation?” but also “What level of evidence should reasonably have been generated by this point in the programme’s lifecycle?” An acute crisis may justify rapid assessment; a mature programme should increasingly be expected to generate stronger causal and comparative evidence where feasible. In that sense, the RCT Purpose Rwanda is now pursuing is not a rejection of proportionality—it is proportionality evolving with the programme. The real challenge is ensuring that urgency explains temporary evidentiary limitations without becoming a permanent excuse for not closing the evidence gap.
I think this captures one of the most important realities of evidence-informed decision-making: inaction is also a decision, and in some frontline contexts, it carries a human cost. When people are already experiencing harm, waiting several years for an RCT before acting may itself be ethically difficult.
I would consider programme monitoring data, lived experience, and practitioner judgement legitimate sources of interim evidence, particularly when they are triangulated rather than treated independently. However, I would distinguish between evidence sufficient to act and evidence sufficient to claim effectiveness. Monitoring data may justify continuing or adapting an intervention, but it should not automatically be interpreted as proof that the intervention caused the observed outcomes. That distinction protects both communities and scientific integrity.
For me, the appropriate standard is therefore proportional, transparent, and progressive. The greater the potential harm and urgency, the more reasonable it may be to act on imperfect evidence—but the decision should be explicit about uncertainty, continuously monitored, and designed to generate stronger evidence over time. As evidence accumulates, the epistemic standard should rise rather than remain permanently at the interim level.
Your closing phrase, “Individually weaker, collectively stronger,” captures this well. The challenge is not choosing between practice and rigorous research; it is building systems where frontline learning informs immediate action while those same experiences progressively generate the stronger evidence needed to guide future decisions.
I believe programme monitoring data, triangulated with lived experience, practitioner judgement, and the best available research, provide a legitimate basis for interim decision-making when delays could cause harm. In tuberculosis control, for example, we cannot postpone action while waiting for definitive evidence when patients face diagnostic delays, stigma, and barriers to accessing care. However, decisions should be proportionate to the potential risks, guided by established clinical and ethical standards, and accompanied by continuous monitoring, transparent documentation of uncertainty, and referral where necessary. The key is not to treat preliminary evidence as conclusive, but to use it responsibly while generating stronger evidence in parallel. Urgency should accelerate learning, not lower our commitment to scientific rigour.
Individually weaker, collectively stronger.