“Monitoring and evaluating our projects is the most important part of our work, we need to understand if something has performed as anticipated, or if not, an understanding of why. ” 

That’s the view of Sarah Yarrow, National Manager of Living Water, a partnership between the Department of Conservation (DOC) and Fonterra. The partnership has been trialling solutions to improve freshwater quality in five catchments around New Zealand since 2013. Yarrow says it’s important to monitor and evaluate the trials to learn whether the outcomes (targets) have been achieved, the cost involved, and whether the intervention being trialled could be scaled and applied more widely. 

Living Water’s work involves implementing interventions to determine if they improve water quality. One example is the trial of detention bunds in the Wairua River catchment in Northland to assess whether they reduce sedimentation in waterways. This type of work is known as a ‘proof-of-concept’ programme, where ideas for improving water quality are tested to see if the concepts work in the real world. 

Yarrow says both Living Water partners (DOC and Fonterra) need evidence of why concepts work so they can be understood, promoted and used on farms and in catchments across New Zealand. That led to the decision to make monitoring and evaluation a core component of Living Water’s work. 

Monitoring and evaluation is a process whereby actions and activities (in this case of an organisation: Living Water) are recorded and measured, and then assessed and interpreted to see if the desired outcomes have been achieved. Monitoring and evaluation can take many forms. The monitoring may include measuring the outcomes of trials, for example fish surveys, so the results can be assessed against baseline data to see if progress has been made towards targets. 

In addition to measuring and assessing the outcomes of trials, the organisational partnerships can be monitored and assessed to evaluate how they are working and if they are producing the desired social and cultural outcomes. The monitoring and evaluation takes place within a structured framework to ensure the information gathered in the process is useful to the participants. 

The monitoring and evaluation framework was developed in 2016 over eight months in workshops with partners in each catchment says Yarrow. 

“Living Water wanted to know what information would be most useful and meaningful for partners”, says Yarrow. “By being involved in the development of the framework, stakeholders understood that Living Water was primarily focussed on trialling concepts rather than restoring catchments, though many trials, such as planting and habitat restoration, have also improved the catchments.” 

An overall national framework of outcomes and targets was developed, along with each catchment developing a targeted plan. For example, in Wairua a primary issue is sedimentation, so the trials (and the monitoring) are focussed on concepts and interventions to reduce sediment entering waterways. In the Pūkorokoro-Miranda catchment, activities are centred around a community-led ‘mountains to sea’ restoration project. Monitoring in Pūkorokoro-Miranda focusses on community participation in projects. There is no comparison undertaken between catchments, as each is individual and unique. 

The environmental impact of the trials is being measured in all catchments. While it may take years, even decades to make significant improvements to water quality, there are short-term results that can be measured to provide indications of trends. For example, fish surveys are a good indicator of the impact of habitat restoration.  

In addition to assessing the environmental impact of the trials, Living Water is also assessing the performance of its partnerships and looking at the social and institutional arrangements that might be needed to bring people, groups and organisations together in the right way to address declining water quality. 

“Water quality is really a people problem”, says Yarrow. “It’s the actions of many people over many years that have led to the current state of New Zealand’s waterways and lakes. Consequently, people, organisations and the way we operate within society will need to change to improve water quality outcomes.” 

Living Water recognised the importance of working in partnerships to enable people to understand what was happening in their catchments and allow them to participate in the trials so they could see firsthand what works and what doesn’t. The impact of this type of partnership is being monitored and evaluated to see if participation in Living Water has changed attitudes and behaviours. 

Lastly, the economic impact of the trials is being monitored and assessed. That includes the actual cost of the interventions, such as building a detention bund, and also the impact on the financial viability of farming where interventions lead to changes of land use or farming operations. Yarrow says environmental gains shouldn’t necessitate farming losses. 

Monitoring and evaluation allows the results of trials to be collected, understood and shared. Even when a trial is inconclusive, monitoring enables us to gain an understanding of why it may not have achieved the desired outcomes and inform future trials. Yarrow says the monitoring and evaluation programme also is providing evidence of how to affect change. 

“Living Water is working in partnerships to empower people to make changes to improve water quality”, says Yarrow. “It’s important to understand how to motivate people and harness their collective energy to make change. For example, the trial of detention bunds in Northland is as much about trialling how to educate landowners about the problem of sedimentation and how detention bunds could address that issue, as it is about building bunds to see if they work effectively.” 

By recording ‘how’ landowners were engaged in the trial and their response to it, Living Water has learned how to ensure this type of trial will be understood and supported by affected landowners. 

Yarrow says monitoring and evaluation is fundamental to Living Water. 

“It shines a spotlight on what and how things are done and what’s being achieved”, says Yarrow. “Without a monitoring and evaluation framework in place there would be no way of knowing what was or wasn’t working and why. Measuring organisational performance in addition to the outcomes of trials ensures the role of the organisation and partners is also understood, as the degree of participant buy-in can be critical to the success of a project.” 

As an example Yarrow says riparian planting events can be more successful if well supported by local communities. It’s important to understand what makes local communities choose to support these events, so the conditions and approach can be applied elsewhere to achieve similar positive outcomes. Yarrow says anyone engaged in partnership projects, or trials where the plans, actions and outcomes need to be recorded, measured and assessed would benefit from developing a monitoring and evaluation framework before commencing work.