Home› Groups› Evidence Informed Decision Making(EIDM)› Evidence Informed Decision Making(EIDM)› When the Evidence Isn't Ready Yet: How Should…

When the Evidence Isn't Ready Yet: How Should Practitioners Make Decisions in the Meantime?

Started by Charles Sep 1, 2026 7 replies 👁 7 views
Charles Member Community Champion (1,500+ points) Community Champion
Sep 1, 2026 at 10:48 am

Most discussions of evidence-informed decision making assume a sequence:
generate rigorous evidence, then let it inform decisions. In practice, for many
of us working directly with communities, the sequence is inverted. Decisions
cannot wait for evidence to mature, people are in front of us now, and the
evidence base we would ideally want is still years away.

 

At Purpose Rwanda, we have built a peer-led addiction recovery model, the
Purposeful Agent of Transformation (PAT) approach, that has produced a 70%
sustained recovery rate across 1,700+ beneficiaries in 5 districts since 2017.
That figure comes from programme monitoring data, not a randomised controlled
trial. We are now pursuing a more rigorous RCT design specifically to test what
we believe we already know from years of direct practice. In the meantime, we
have continued making real-time decisions, who to enrol, how to structure peer
mentorship, when to refer upward to clinical care, using evidence that would
not satisfy a strict EIDM standard, because the alternative was to make no
decision at all while people remained in crisis.

 

This raises a genuine question for this community: what is the appropriate
epistemic standard for decisions that cannot be deferred? Is programme
monitoring data, triangulated with lived experience and practitioner judgement,
a legitimate basis for interim decision-making while more rigorous evidence is
pursued in parallel? Or does treating it as sufficient risk normalising weaker
evidence simply because urgency is real?

 

I suspect most of us in frontline or applied research contexts have quietly
developed our own working answer to this tension, even where our institutions
have not formally named it. I would value hearing how others here have
navigated the gap between the evidence we have and the evidence we wish we had,
particularly in contexts where waiting has a real human cost.

 

Individually weaker, collectively stronger.

 

 

Dr. Magoba Member Expert (800+ points) Expert
2 months ago

Charles, this is an important and very practical reflection. I strongly agree that in frontline public health, evidence-informed decision-making is rarely as linear as “generate evidence first, then make the decision.” When people are already experiencing harm, waiting for perfect evidence can itself become a decision—with potentially serious consequences.

I would argue that programme monitoring data, lived experience and practitioner judgement can legitimately support interim decisions, provided we are transparent about the level and limitations of the evidence. The key distinction, in my view, is between using imperfect evidence responsibly and presenting imperfect evidence as definitive proof.

For me, the appropriate approach is proportionality: the urgency and potential consequences of the decision should determine how much uncertainty we are willing to tolerate. Where the human cost of waiting is high, the best available evidence can guide action, while stronger evaluation—including your planned RCT—continues in parallel. Monitoring data can help us understand reach, retention, implementation fidelity, emerging outcomes and unintended effects, while lived experience helps explain what the numbers alone cannot.

The important safeguard is to build learning into implementation: act, monitor, triangulate, reassess and adapt. In that sense, “evidence-informed” should not necessarily mean “evidence-perfect.” It should mean making the most defensible decision possible with the evidence available, being explicit about uncertainty, and remaining willing to change course as better evidence emerges.

Your PAT experience illustrates this tension beautifully: practice generates learning, learning informs adaptation, and rigorous research can subsequently test whether those observed effects are causal and generalisable. Perhaps the real challenge is not choosing between practice and rigorous evidence, but creating systems where both can continuously inform each other.

Individually weaker, collectively stronger.

jeff Member Newcomer (0+ points) Newcomer
2 months ago
Jeff,
Good morning, Charles. I think the United Nations model fits best in
this context, particularly because it provides a framework that can
accommodate the demands of a rapidly evolving crisis situation. Where a
crisis requires real-time solutions and immediate decision-making, there
should be room for a rapid assessment that can generate timely and
relevant information for action. In such circumstances, relying solely
on routine monitoring data may not be sufficient, as monitoring systems
are often designed to capture trends over time rather than provide
immediate insight into an emerging situation. It may therefore be more
appropriate to prioritize fresh, compelling, and context-specific
evidence that reflects the current realities of the crisis. A quick
assessment can help gather this evidence within a short timeframe,
identify the most urgent needs and gaps, and provide decision-makers
with the information required to respond effectively. This approach can
ensure that interventions are based on current realities while still
maintaining sufficient rigor to support sound and timely decisions.
Charles Member Community Champion (1,500+ points) Community Champion
2 months ago
Dr. Magoba and Jeff, reading your two responses together, I think you may have identified two genuinely different evidence problems that this thread has been treating as one.
Dr. Magoba's proportionality framework, tolerate more uncertainty where urgency and stakes are high, describes what I would call the steady-state version of this tension: an ongoing programme like ours, where monitoring data accumulates over years and the "evidence we have" grows more defensible over time, even without a completed RCT.
Jeff's UN rapid-assessment model describes something structurally different: an acute crisis, where routine monitoring is the wrong tool entirely because it was never designed to capture what is happening right now. A rapid assessment is not weaker evidence gathered under time pressure, it is a different evidence-generation method suited to a different temporal problem.
This distinction matters because it exposes a risk neither of you named directly: if organisations and institutions do not clearly separate these two situations, "urgency" can quietly become a permanent justification for lower evidence standards, even in programmes that are no longer in crisis and have had years to build something more rigorous. Proportionality, taken seriously, should include an honest accounting of how long a programme has actually been operating under "emergency" evidentiary standards, and why.
For us at Purpose Rwanda, this is a fair challenge to sit with. We are past the acute crisis phase Jeff describes, we have had years to build stronger evidence, and the RCT we are now pursuing is, in a sense, evidence that proportionality's clock has run out for us specifically. I suspect that clock looks different for every organisation in this thread. Perhaps the harder question is not proportional to what, but proportional for how long.
Dr. Magoba Member Expert (800+ points) Expert
2 months ago

I think this is an important distinction, and I particularly agree with your question: “proportional for how long?” Proportionality should not become a permanent exemption from stronger evidence.

My framework is useful because it recognizes that evidence requirements should reflect urgency, risk, feasibility, and the consequences of delay. However, proportionality must also be dynamic. As a programme matures, monitoring systems improve, implementation experience accumulates, and resources become available, the justification for relying primarily on lower-certainty evidence should progressively weaken. Otherwise, an emergency standard can become institutionalized as the default.

I would therefore add a third dimension to the framework: evidence maturity. The question should not only be “What evidence is proportionate to the situation?” but also “What level of evidence should reasonably have been generated by this point in the programme’s lifecycle?” An acute crisis may justify rapid assessment; a mature programme should increasingly be expected to generate stronger causal and comparative evidence where feasible. In that sense, the RCT Purpose Rwanda is now pursuing is not a rejection of proportionality—it is proportionality evolving with the programme. The real challenge is ensuring that urgency explains temporary evidentiary limitations without becoming a permanent excuse for not closing the evidence gap.

Dr. Magoba Member Expert (800+ points) Expert
2 months ago

I think this captures one of the most important realities of evidence-informed decision-making: inaction is also a decision, and in some frontline contexts, it carries a human cost. When people are already experiencing harm, waiting several years for an RCT before acting may itself be ethically difficult.

I would consider programme monitoring data, lived experience, and practitioner judgement legitimate sources of interim evidence, particularly when they are triangulated rather than treated independently. However, I would distinguish between evidence sufficient to act and evidence sufficient to claim effectiveness. Monitoring data may justify continuing or adapting an intervention, but it should not automatically be interpreted as proof that the intervention caused the observed outcomes. That distinction protects both communities and scientific integrity.

For me, the appropriate standard is therefore proportional, transparent, and progressive. The greater the potential harm and urgency, the more reasonable it may be to act on imperfect evidence—but the decision should be explicit about uncertainty, continuously monitored, and designed to generate stronger evidence over time. As evidence accumulates, the epistemic standard should rise rather than remain permanently at the interim level.

Your closing phrase, “Individually weaker, collectively stronger,” captures this well. The challenge is not choosing between practice and rigorous research; it is building systems where frontline learning informs immediate action while those same experiences progressively generate the stronger evidence needed to guide future decisions.

Dr. Magoba Member Expert (800+ points) Expert
4 weeks ago

I believe programme monitoring data, triangulated with lived experience, practitioner judgement, and the best available research, provide a legitimate basis for interim decision-making when delays could cause harm. In tuberculosis control, for example, we cannot postpone action while waiting for definitive evidence when patients face diagnostic delays, stigma, and barriers to accessing care. However, decisions should be proportionate to the potential risks, guided by established clinical and ethical standards, and accompanied by continuous monitoring, transparent documentation of uncertainty, and referral where necessary. The key is not to treat preliminary evidence as conclusive, but to use it responsibly while generating stronger evidence in parallel. Urgency should accelerate learning, not lower our commitment to scientific rigour.

Individually weaker, collectively stronger.

Rhoda Nakhosi Admin Community Champion (1,500+ points) Community Champion
4 weeks ago
@Charles Bawate Bawate, "proportional for how long" is the most needed question, and I notice it landed hardest on your own programme, which is a rare kind of honesty in a discussion like this. But I want to test the frame itself. Your distinction between steady state and acute crisis assumes an organisation knows which one it is in. In my experience, many programmes never formally exit "crisis" because the emergency was never declared in the first place, it was simply the conditions they started under. No one wrote down a start date for the clock you are describing, so no one can say when it has run out.
That makes me wonder whether "proportional for how long" is answerable at the organisational level at all. If the clock only starts when someone names it, then the discipline you are asking for may need to sit with funders or regulators rather than with programmes themselves, because the programme under pressure will always find a reason the emergency is not over.
Where do you think that accountability should actually sit?