Weekly WASH Graph: a sneak preview at measuring system strength

Published on: 03/09/2018

Whilst many in the WASH community were at the Stockholm World Water Week, I had a quiet time at the office, finalising a report with the baseline situation of the strength of the WASH system in one of IRC's focus countries: Honduras. In IRC's strategy 2016-2030, we commit ourselves to strengthening the WASH system both at national and district/municipal levels in a number of focus countries, including Honduras. With this commitment comes also the need to monitor progress in the strength of the WASH system and IRC's contribution to that improvement. Over the past year, we have therefore worked hard to develop a monitoring framework and to establish a baseline. In order to do so, the following main questions were raised: how do we define the WASH system? And how can we measure its strength?

Defining the WASH system and its building blocks

As outlined in this recently published working paper by my colleagues Angela Huston and Patrick Moriarty, IRC defines the WASH system as all the people, components and functions that are needed to deliver WASH services. It includes all the actors (people and institutions) and all the factors (infrastructure, finances, policies and environmental conditions) that affect and drive the system. These factors are many, and they interrelate with each other, giving the WASH system the characteristics of a complex adaptive system. In order to deal with this complexity, IRC has identified nine building blocks (see figure 1), each of which can be understood as a sub-system of the overall WASH system. Each building block includes the actors and factors that must work together to perform a function or series of functions; the same actor may contribute to the functioning of multiple building blocks. The boundaries of these building blocks are not absolute and there are probably other ways to define the building blocks. But we found that these resonate with the people and organisations we work with, and allows us to group factors and actors in a meaningful manner. Moreover, they facilitate prioritisation of goals by helping identify which specific building blocks need strengthening, or rather need consolidation.

Figure 1: the nine building blocks that form the WASH system (Huston and Moriarty, 2018)

Measuring the strength of the WASH system

One of the main applications of the building blocks is measuring the strength of the WASH system. We do this by answering a number of questions (three to five per building block) around some of the key elements that need to be in place to ensure that each building block is strong. Depending on whether such an element is in place or not, a score is assigned to that question on a scale of 1 (non-existent) to 5 (very strong). For example, for the building block on monitoring, the following five questions need to be answered and scored on that scale.

• There is a national monitoring framework, through which different sectoral monitoring systems speak to each other
• The monitoring systems include service delivery indicators, on service levels, service provider performance and service authority performance
• The monitoring systems actively cover the entire country
• The data from the monitoring system are analysed and used at sector level, e.g. for macro-level planning, trends analysis, policy making
• The data in the national monitoring system are regularly updated

The total score of each of the building blocks is calculated as the average of the question scores. In addition to scoring the building blocks, a narrative and analysis is also provided to justify why a particular score was assigned.

The overall strength of the system is then defined by the scores of the nine building blocks. We do not take an average across these scores or add them up. That would deliver a number that would not provide much meaning. More valuable is getting insights about which building blocks are strong and which ones are weak, and above all what the entire set of scores means. Since the boundaries of the building blocks are not fixed and the building blocks are interrelated if the building block on finance scores low, it is probable that it will also affect several of the other ones. This exercise is done for four sub-sectors; water, sanitation, hygiene and WASH at extra-household settings (school and health centres), and at national and district level.

Weekly WASH Graph: the strength of the WASH system in Honduras

This week's Weekly WASH Graph gives you a sneak preview of the results of measuring the strength of the building blocks with this methodology, using Honduras as an example (Table 1). It shows the scores of each building block and the corresponding colour code for the sub-sectors of water supply and sanitation at national level.

Table 1: scores of the strength of the nine building blocks of the WASH system in Honduras

As illustrate above, some of the building blocks are quite strong, particularly policy and legislation and infrastructure development. Honduras has a clear legal framework for WASH service delivery and corresponding policies. It also has a long track record in the development of WASH infrastructure and has a whole set of procedures and guidelines for that. It scores lower in some of the building blocks for long-term service delivery, such as infrastructure management and monitoring. It is not that there are no frameworks in place for operation and maintenance, asset management, or monitoring; it is more that they are not applied fully, neither covering the entire country, nor being fully utilised. The lowest score is for finance. The financial framework is weak, with only very broad definitions of where the financing for the various life-cycle costs needs to come from and little definition on the funding mechanisms for the sector. Finally, it can be seen that in some of the building blocks the water supply and sanitation sub-sectors score the same (e.g. in planning) as they are generally treated as being in same sector. In others: there are some differences; for example, the monitoring of water supply is stronger than for sanitation.

Different methods, same message

We at IRC are not alone in trying to measure the strength of the system and this is surely not the only methodology. The World Bank in its Country Status Overview paper series, assessed countries against nine building blocks, split into three groups (enabling, developing and sustaining services). More recent work by the World Bank uses five building blocks to assess the strength of the enabling environment, first in 16 countries across the globe and then in seven countries in the Danube region. UNICEF has been developing and using the WASH-BAT (Bottleneck Analysis Tool), another tool to score the strength of some seven building blocks and to identify barriers to their progress.

These methods have many commonalities, including the definition of the building blocks and approach to scoring. Most importantly though, they also jointly acknowledge the aim to provide a comprehensive overview of the strengths and weaknesses of the WASH system. This way the building blocks most crucial to strengthening the the overall WASH system can be identified and prioritised. And that is probably the key message of this sneak preview: if you want to be able to strengthen the WASH system, you need to have a method of identifying which are the key building block that need strengthening, and you need to be able to track their strength over time. 

Watch this space as we finalise the baselines over the coming months to see both the results of measuring the strength of the WASH system and the reflections on the method for doing so.