Back in 2017, Living Water started exploring the development of a more comprehensive outcomes-based monitoring and evaluation framework.  As a “proof of concept” programme operating in a highly dynamic context, Living Water saw the need to look beyond more ‘traditional’ environmental programme monitoring - focussed almost exclusively on technical indicators and output targets that often ignored the important influence and impact that people and organisations have on achieving positive change. Until recently, the performance of many catchment programmes has been judged largely on inputs, activities and outputs. This approach has increasingly been questioned as being too concerned with efficiency, without a corresponding focus on effectiveness and what lessons and benefits are actually arising from programme, funding and activities.

Developed by the Living Water team with support from Viv Sherwood, Will Allen, and Toni White, the Framework is more focussed on outcomes (and impacts) related to people as well as the environment. These outcomes include both social outcomes as contributions to impact (e.g. learning, understanding, perceptions, attitudes and behaviors), as well as longer term biophysical, ecological, social or economic changes in these catchments.  The logic model is underpinned by people-centred causal assumptions, and a mix of indicators, both quantitative and evaluative (qualitative). See the National Planning, Monitoring & Evaluation Framework.

Now as Living Water nears the end of its 10 year programme, Will Allen and Viv Sherwood have returned to help Living Water with its evaluation.  “We’re working together with the key people involved to put together a range of performance stories. These include reflecting on the achievements, key lessons, and recommendations, developed through a combination of interviews and workshops”, says Allen.

While this evaluation process is still underway, there are some things that people are noticing across these and other programmes that they work in, says Allen, and are the sorts of things that people recognise will be useful in future initiatives.

• Programmes need to be seen as being about relationships (and processes) as well as activities, and both need to be equally planned, resourced and reviewed.
• Partnerships take time and need to be developed with a clear expectation of a shared outcome, even though the exact shape of that outcome may not be specified in advance. Between larger organisations particularly too, partnerships work better when there are multiple connections between partners, not just one individual relationship.
• Staffing requires both “hard” (technical) and “soft” (social) skills and capacity to support real engagement with iwi, hapu and communities, as well as across “disciplines” and departments.

Throughout the 10 years of the Living Water programme, there has been a strong emphasis on documenting work and sharing insights along the way, view all projects here and our stories here providing a wealth of information about what has been done.  The evaluation work is building on this by helping people not only reflect on achievements, but also on lessons learned, barriers and opportunities from this work. These reflections are essential to help those involved focus on the importance and significance of the outcomes, and how to take the lessons forward into the future.

Head to the Monitoring and Evaluation Project Page.