What is monitoring and evaluation?

‘Monitoring and evaluation’ is a process whereby actions and activities (in this case of an organisation: Living Water) are recorded and measured, and then assessed and interpreted to see if the desired outcomes have been achieved. Monitoring and evaluation can take many forms. The monitoring may include measuring the outcomes of trials, for example fish surveys, so the results can be compared with baseline data (collected prior to actions being undertaken) to see if progress has been made towards targets. In addition to measuring and assessing the outcomes of trials, organisational partnerships can be monitored and assessed to evaluate how they are working and if they are producing the desired social and cultural outcomes. The monitoring and evaluation takes place within a structured framework to ensure the information produced from the process is useful to the participants. 

Why is monitoring and evaluation important?

Living Water’s work involves running a series of trials of interventions to determine if they result in improved water quality in the real world. One example is the trial of detention bunds in Wairua, Northland to assess whether they reduce sedimentation in waterways. This type of work is known as a ‘proof-of-concept’ programme, where ideas for improving water quality are tested to see if the concepts work in the real world. 

It’s important to monitor and evaluate the trials to learn whether the outcomes (targets) have been achieved, the cost involved, and whether the intervention being trialled can be scaled and used more widely. Both Living Water partners (DOC and Fonterra) need evidence on the effectiveness of trials so they can be understood, promoted and used on farms and in catchments across New Zealand. 

What’s being monitored and evaluated?

The environmental impact of the trials is being measured in all catchments. While it may take years, even decades to make significant improvements to water quality, there are short-term results that can be measured to provide indications of trends. For example, fish surveys are a good indicator of the impact of habitat restoration.  

In addition to assessing the environmental impact of the trials, Living Water is also assessing the performance of its partnerships and looking at the social and institutional arrangements that might be needed to bring people, groups and organisations together in the right way to address declining water quality. That’s because water quality is really a people problem. It’s the actions of many people over many years that have led to the current state of New Zealand’s waterways and lakes. Consequently, people, organisations, and the way we operate within society will need to change to improve water quality outcomes. Living Water recognised the importance of working in partnership with people to enable them to understand what was happening in their catchments and allow them to participate in the trials so they could see firsthand what works and what doesn’t. The impact of this type of partnership is being monitored and evaluated to see if participation in Living Water has changed attitudes and behaviours. 

Lastly, the economic impact of the trials is being monitored and assessed. That includes the actual cost of the interventions, such as building a detention bund, and also the impact on the financial viability of farming where interventions lead to changes in land use or farming operations. Environmental gains shouldn’t necessitate farming losses. 

How was the monitoring and evaluation framework developed?

The process of developing a monitoring and evaluation framework began in 2016, after Living Water set a new strategic direction for the programme. The framework was developed over eight months in workshops with partners in each catchment. Living Water wanted to know what information would be most useful and meaningful for partners as the programme of trials was rolled out. By being involved in the development of the framework, stakeholders gained an understanding that Living Water was primarily focussed on trialling concepts rather than restoring catchments (though many of the trials, such as planting and habitat restoration, have improved the catchments). Stakeholder participation ensured the targets set and being measured would have value and meaning for partners. 

Is the monitoring the same in each catchment?

An overall national framework of outcomes and targets was developed, along with each catchment developing a targeted plan. Some monitoring and targets are the same across the catchments, while others are specific to a single catchment. For example, in Wairua a primary issue is sedimentation, so the trials (and the monitoring) are focussed on concepts and interventions to reduce sediment entering waterways. In the Pūkorokoro-Miranda catchment, activities are centred around a community-led ‘mountains to sea’ restoration project. Monitoring in Pūkorokoro-Miranda focusses on community participation in projects. There is no comparison undertaken between catchments, as each is individual and unique. 

How is the feedback from the monitoring being used?

Monitoring and evaluation allows the results of trials to be collected, understood and shared. Even if a trial is inconclusive, monitoring enables an understanding of why it may not have achieved the desired outcomes and could inform future trials. The monitoring and evaluation programme also is providing evidence of how to affect change. Living Water is working in partnerships to empower people to make changes to improve water quality. It’s important to understand how to motivate people and harness their collective energy to make change. For example, the trial of detention bunds in Northland is as much about trialling how to educate and inform landowners about the problem of sedimentation and how detention bunds could address that issue, as it is about simply building the bunds to see if they work effectively. By recording ‘how’ landowners were engaged in the trial and their response to it, Living Water has learned how to ensure this type of trial will be understood and supported by affected landowners. 

Why does monitoring and evaluation matter?

Monitoring and evaluation is fundamental. It shines a spotlight on what and how things are done and what’s being achieved. Without a monitoring and evaluation framework in place there would be no way of knowing what was or wasn’t working and why. Measuring the organisational performance in addition to the outcomes of trials ensures the role of the organisation and partners is also understood, as the degree of participant buy-in can be critical to the success of a project. For example, riparian planting events can be more successful if well supported by local communities. It’s important to understand what makes local communities choose to support these events, so the conditions and approach can be applied elsewhere to achieve similar positive outcomes. 

Who could use a monitoring and evaluation framework?

Anyone engaged in partnership projects, or trials where the plans, actions and outcomes need to be recorded, measured and assessed.