Measuring Impact: Monitoring & Evaluation
Aug 23, 2021
M&E refers to the processes by which we can track and assess the progress of a programme or project. Monitoring refers to the routine, regular tracking of progress, whereas an evaluation refers to the systematic assessment of the value or success of a programme.
Why is M&E crucial in mental health programmes?
To understand if a programme or intervention is being implemented as planned
To identify implementation challenges
To assess the extent to which a programme or intervention is achieving its intended objectives
To obtain data to show impact, crucial for securing funding
Thus, M&E of mental health programmes is vital to the sustainability of a programme or intervention. M&E systems can therefore help implementers improve their work on the ground and also demonstrate their achievements. However, setting up a M&E system can be a complex task, and one that requires plenty of time, funds and capacity within the team, which small organisations working at the grassroots may not always have.
As part of our work in impact at Ember, we have supported organisations in understanding their needs with regards to M&E, and where appropriate, facilitated the process of setting up a M&E system. It’s best to talk through the following points with the partner team to assess their existing M&E systems.
Key Thinking Points
1. Ensure there are clearly defined, accurate, and achievable objectives
Grassroots mental health programmes are often ambitious about their goals, as they are starkly aware of the great need for MH support within their communities. However, this can bring about issues when measuring the success of a programme. For example, organisations often set “improving mental health in a country” as their objective. However, many organisations only have the capacity to work in a particular municipality or neighbourhood, or support specific members of a population, so setting a smaller scale objective that reflects this is more realistic and achievable. We often encourage organisations to be more specific and also consider what they can achieve given the resources they have available.
2. Define measurable targets and indicators
Targets and indicators are used to measure the degree to which objectives are being achieved. Importantly, they also allow the identification of what data needs to be collected, from whom and with what frequency. The process of defining targets and indicators is useful to set priorities, particularly when an organisation delivers a range of programmes and interventions on limited resources. Many times there is a mismatch between what we want and can measure, and identifying the most relevant and feasible targets can be useful.
3. Establish clear and efficient procedures for data collection and analysis
Gathering, processing and analysing data can be hugely resource-intensive processes. In order to make these more manageable, it is important to question what data is collected to measure each indicator, how it is collected, when it is collected and how it will be used. Discussing each of these points from the outset as a team is critical to ensuring that only essential data is collected, through easy to use systems which enable collaboration and efficiency.
4. Familiarise with the tools available for M&E
Fortunately, technology has provided an opportunity to facilitate data collection and analysis. Learning what tools are available and how to operate them in a way that suits the organisation can be tricky but is enormously beneficial, particularly when organisations are short staffed. For example, collecting data through an online tool rather than a paper form can save time, space and allow for automated analysis to be done.
How have we implemented this?
Identifying these issues was important for them to formulate strategies to improve their work and get closer to achieving their intended objectives. M&E systems are essential to understanding an organisation’s progress and impact in supporting the mental health of the communities it aims to serve. For this reason, the thoughtful definition of objectives and indicators is necessary to ensure that the metrics that we use to track and assess a programme or intervention are relevant and appropriate for them and the communities that they serve. Setting up an M&E system that works well can facilitate the process of conducting an evaluation later on, details of which we will cover in our next blog.