Programme Evaluation, Impact Assessment, and Impact Evaluation: Commonality and Differences

Organizations are often unable to demonstrate success, and worst, unknowingly continue supporting ineffective programmes. How do you know whether you planned the programme right? could you have been more efficient? Did you target your programme correctly? What are the impacts of the programme? Answering these types of questions is largely known as ‘Evaluation’.

However, often similar-sounding names or terms for ‘types’ of evaluation cause great confusion among implementers, policymakers, and even M&E professionals. Let’s explore commonality and differences between the terms – programme evaluation, impact assessment, and impact evaluation.

The Commonality and Differences

1. IMPACT EVALUATION

Impact Evaluation answers a causal parameter – change in an outcome caused by the intervention above-and-beyond other interventions or development changes. To prove causality, we must answer a counterfactual question – what would have happened if ‘this’ programme did not exist? For counterfactual analysis, we need a control group which is identical to the programme group except for the intervention(s) itself. Impact evaluation calls for rigorous economic, statistics and epidemiology methods to find a control group and then find attributable impacts of a programme. While impact evaluations should ideally be planned before the programme begins, it can only be completed after the intervention concluded, and thus, can inform future phases or variants of the programme.

2. PROGRAMME EVALUATION

Programme evaluations ask a broader set of questions and can be considered before-during-after a programme or intervention. Programme evaluation is often done using mixed methods (qualitative and quantitative), and the rigor or methods applied is tailored to each organization’s and programmes unique learning needs. NEERMAN recommends using OECD-DAC criteria to organize the evaluation question and methods. If your organization does not know what type of evaluation is needed, this framework will guide you in asking the right questions.

3. IMPACT ASSESSMENT

Impact Assessment is sometimes erroneously used to mean impact evaluation or programme evaluation. However, it is basically a mixed-methods approach to answer questions related to one of the six OECD-DAC criteria called impact. The effectiveness criterion asks, ‘whether the programme delivered the objectives it was accountable for?’ whereas the impact criterion goes beyond and asks, ‘the extent to which the programme has/will deliver higher-level effects’. Impact assessment can be done before the programme begins and should assess even unintended or negative outcomes. Proving causality is not a prime intent of impact assessments but proving and assessing the theory of change is. However, if a causal question is asked under the ‘impact’ criterion, then impact assessment can mean ‘impact evaluation’ in this context.

So, what term should you use?

Our suggestion: Don’t use a term! Instead, explain the purpose of the evaluation, how you intend to use the findings, and example questions you want to ask.

Transform the World with NEERMAN

Do you have a specific project or partnership idea?
Do you need a micro-consulting on M&E questions that Google can’t answer?
Do you have general comments or inquiries not covered above?