Assign sustainability indicators, augmenting the results chain where necessary
Though the results chain is the management and measurement ‘backbone’ of intervention, it is only given meaning if the indicators of change are well formulated, ie SMART (specific, measurable, achievable, relevant and time-bound) and if they are tested and amended accordingly. In some instances, you may need to use proxy indicators to tell you something about a change that is hard to measure.
Reality check: Assigning indicators
Assigning at least one indicator to each box in the results chain helps you to track progress accurately and to examine your assumptions about how change was predicted to take place. For some programmes, this can create a large number of indicators to track. This level of detail can be useful so long as it informs decisions or is used for reporting. To avoid irrelevant indicators, ask yourself:
- Will changes in this indicator inform my next steps?
- Will this indicator help establish the link between change at one level in the results chain and the next level up, ie the chain of causality?
- Is this indicator needed for reporting purposes? It is a common mistake to select indicators at the start of the programme, and then stick with them, regardless. What seems like a sensible indicator during planning might not be practical when it comes to actual measurement, in terms of feasibility, reliability, significance or replicability on a larger scale.
The results chain and its indicators are only meaningful in a market systems development programme if the sustainability of output, outcome, and impact changes are also measured. Assigning sustainability indicators will ensure that your programme takes sustainability seriously.
Assessing sustainability requires you to determine whether the innovation market players have adopted is continuing independently of the programme and whether crowding-in has taken place.
The Systemic Change Framework (see Chapter 4) provides a way of reflecting on both the sustainability and scale of impact and assists programmes in planning their next move.
Sustainability indicators can be created for each element of the framework (adopt, adapt, expand and respond) and can be used to augment pilot and crowding-in phase results chains.
Sustainability indicators help programmes:
- Track changes in partner behaviour, capacity and incentives, drawing attention to those partners who are struggling to continue with new roles
- Track changes in the level of partner ownership, and signs of the shift from adapt to adopt (ie independent actions)
- Assess whether scale and sustainability depend on too few players
- Identify whether other players in the systems are responding positively to changes introduced, or where the wider environment remains unsupportive of pro-poor change
Note that when you measure scale (how many, how much) you focus mainly on the changes stimulated by your interventions. When measuring sustainability, the same is true, but you must also be alert to what has not happened and what is not yet in place.