Is the anti-corruption field making progress in tackling corruption? Debate on whether existing efforts are registering successes is extensive, although it is widely agreed that the field lacks evidence on anti-corruption effectiveness. Monitoring and evaluation (M&E) can contribute key evidence but has often been overlooked in these debates. This report starts to address this gap by mapping and comprehensively reviewing the quality of the evaluation evidence available to the field.

Over the last two decades international development organisations implementing anti-corruption reforms have invested increasing levels of resources into M&E. This is with the stated goals of helping learn about and improve outcomes from their work, while simultaneously ensuring accountability to stakeholders. In a context where development organisations are under ever greater pressure to understand their impact, the role of M&E has never been more critical.

Evaluating anti-corruption interventions is nonetheless challenging. Change is hard to observe and measure. Political, economic, and socio-cultural factors have critical influences on outcomes. To help overcome these challenges, there is a small body of good practice guidance for practitioners on how to evaluate anti-corruption interventions. However, until now, the existing evaluative work produced has not been subject to detailed review.

A new framework for assessing evaluation quality

The report is organised around a new framework for assessing the quality of anti-corruption evaluations, which is informed by complexity theory in the evaluation field and is divided into two parts:

• The first assesses the extent to which programmes are set up to support strong M&E.
• The second assesses the quality of evaluation reports themselves, identifying and elaborating a range of criteria relating to evaluation design, methods, measurement of outcomes, gender responsiveness, and potential for wider application of finding.

In total the framework covers 21 data points and applies simple scoring criteria to assess existing evaluations. The framework can be found in the Annex.

Employing this framework, the report provides a structured review of 91 evaluation reports for anti-corruption interventions published by 11 development agencies and non-governmental organisations between 2010 and 2023. They cover a wide range of anti-corruption interventions implemented across the globe.

Findings on programme set-up

Many of the challenges around evaluation quality stem from the way anti-corruption programmes are set up. Theory of Change (ToC) – an adaptive approach to planning and evaluating complex development programmes – appears to be weakly implemented and often misunderstood in the field. Only 20% of the evaluations reviewed include a clear ToC (as originally set out by the programme/ portfolio) which outlines how the intervention(s) are expected to contribute to change. Many of these ToCs lack critical elements, such as evidence of grounding in contextual analysis as well as incorporation of assumptions and risks.

Programmes additionally do not appear to routinely collect and/or use monitoring information for evaluation. Of the evaluations reviewed, 68% did not incorporate monitoring information of a qualitative or quantitative form. When programmes use outcome indicators, these are often not suitable for tracking change at the level at which intervention operates; for instance, a country-level index is used to monitor the impact of a targeted intervention. Nearly three quarters of the programmes reviewed also lacked baseline information to form a comparison point against which to understand changes which might have taken place. The implication is that, in many cases, programmes are poorly positioned to optimise an evaluation process.

Findings on evaluation quality

Although there are some high-quality evaluations, there are systematic problems with the quality of the evidence produced through M&E. Around half of the evaluations reviewed aim to assess impact and almost all are asked to assess effectiveness as defined by the OECD. The challenge is that most evaluations are not designed in a way which would allow them to reliably do so. Three quarters lack a formal research design. Several innovative approaches to evaluating complex programmes have also not been attempted in this space.

The use of contextual analysis is an important area of weakness. It is a common mantra that the success of anti-corruption measures depends on the context, and yet just over half of the evaluations did not present any contextual analysis. Where contextual analysis is presented, it is often high-level and not used to situate the findings.

Despite recent advances in measuring corruption, many of the tools the field has at its disposal are not being used in evaluation, a domain where there is high scope for application. Cross-national corruption perception indices and surveys of programme partners are the main measurement tools used, while recent innovations in measuring corruption risk and institutional defences against corruption are under-employed.

The review includes some useful examples of how evaluators have navigated challenges in evaluating anti-corruption work. Overall, however, several other common weaknesses across these evaluations are highlighted, including limited attentiveness to gender and intersectionality; lack of attribution of evidence; and failure to consider potential for the wider application of findings.

Addressing constraints around M&E

The findings are not surprising when considering some of the constraints around evaluation practice. The median time frame given to external consultants to complete their work is three months – this is not a lot of time to understand a complex programme and produce independent analysis. Evaluation also usually only starts at the end of a programme’s lifespan. There is a logic to this, of course, in that to fully understand its contribution a programme needs to be finished but this means many opportunities for applying learning have passed.

The report concludes with some high-level recommendations to organisations commissioning and managing evaluations. Several of these relate to the structural constraints identified. More broadly there is a call to make more use of the tools it already has its disposal for understanding change related to anti-corruption interventions. It is hoped the report prompts wider debate on how anti-corruption practitioners can better realise value from M&E and improve understanding of their impact.