Measuring Our Impact

The Southern Africa Trust has invested substantial resources in its monitoring and evaluation (M&E) system. Monitoring of outputs occurs on a monthly and quarterly basis using the various dashboards and quarterly logframe report.

Monitoring of outcomes takes place on an annual basis through the annual review and the annual M&E report which is based on the logframe. Evaluation takes place every three years based on the findings of the baseline survey instrument and the interim impact assessment report (an internal evaluation tool) and the external evaluation.

The entire organization is familiar with the various instruments, all of which have been integrated in the strategic planning and management decision making processes. This ensures monitoring is not just undertaken to meet donor requirements but is a strategic management tool that is integrated into the day to day way of working of the Trust.

Instrument

Frequency

Questions Addressed

Use

Management Dashboard

Monthly Is the Trust on track with its plans or are there any areas management should be concerned about? Used by management to check overall running of the organization

Logframe Report

Quarterly Is the Trust delivering its planned outputs and outcomes, to standard and within budget? What indicators are there to show this? How do these contribute to reaching the overall purpose of the Trust? Used by management team, board and donors to review progress

Governance Dashboard

Quarterly Is the Trust on track with its strategic outcome and institutional development plans or are there any areas the Board needs to be concerned about? Used by the Board to check overall running of the organization

Annual Monitoring Report

Annual Has the Trust delivered its planned outputs, to standard, and what is the initial indication of whether or not the planned outcomes were achieved over the annual period? Helps the team think about why they are doing the various activities and how they are doing them. Used by management to review progress, check outcomes are likely to be achieved and highlight areas where further research or new M&E instruments are needed.

Annual Review (With Donors And Independent Reviewers)

Annual Has the Trust met its planned outputs and outcomes for the year? How is the Trust’s purpose being achieved? Are the outputs effective and efficiently delivered, and are the planned outcomes still relevant? Are the Trust’s strategies appropriate? Is the Trust delivering value for money? What case studies are there to demonstrate this? Helps the team think about why they are doing the various activities and how they are doing them, with a view to adapting its approach in the forward period. Used by donors to monitor progress and by the Trust to reflect on its practice

Knowing Civil  Society Organisations Report

Annual Or Every 2 Years What is the profile and organizational health of the Trust’s grant partners in relation to the capacity building, operating environment, and other programme focus areas of the Trust and what does this tell the Trust about the primary sector with which it works? Packaging available information from grant partners as one window of information on the state of the sector. Used to inform the Trust’s learning about changes, strengths, and weaknesses in the sector and to review its strategies.

Baseline Study And Follow Up Studies In Year 3 And 5

Twice Every 5 Years What is the general knowledge base, skills levels, organisational health and policy work output of the Trust’s primary sector and how has this changed over the period of the Trust’s interventions? To monitor changes in the state  of the sector in key outcome focus areas of the Trust’s work so as to provide quantitative and data on the impact of the Trust’s work

Interim Impact Assessment

Every 3 Years What impact is the Trust having? What case studies are there to demonstrate this impact? What strategies has the Trust used to achieve this impact and how have they worked or not worked? A self-assessment with the assistance of an external M&E advisor. To determine, through quantitative and qualitative data, whether the Trust is making the difference that it set out to achieve (i.e. whether its strategies are working) and alert management and Board to any areas where outcomes might not be met, and to collect data for external evaluation

External Evaluation

Every 3 Years What is the impact, effectiveness, relevance, and efficiency of the Trust’s work as a whole? How should it be improved in the future? Is the purpose and mission of the Trust still relevant in the context in which it works? External assessment of the Trust’s work and reason for existing.

Participatory and Informal M&E Mechanisms

With the exception of the management and governance dashboards, the methodology for using all the other M&E instruments includes extensive consultation with the Trust’s grant partner organizations and other non-grant programme / strategic partners through various mechanisms.

In addition, in the implementation of our programmes, we have ongoing consultation with the various groups that have a stake in the Trust’s work about the approach, way of working, and relevance of the Trust – through bilateral discussions with key partners, an ongoing programme of structured national consultations with a wide range of civic formations in groups in all the countries in which we work, and occasional regional civil society consultations (either in general about the work of the Trust or specifically in relation to one programme area or key initiative). These more informal M&E processes are a very important way in which we test the effectiveness of our strategies and learn about how to do things better.

Much less significant but equally important: we do a thorough evaluation against the objectives set for every event (convening, policy dialogue, product launch, awards, etc.) that we host, which is a very useful way of demonstrating how M&E concerns have been integrated into our day-to-day work. This includes documented feedback from participants in the event.

Comments and Conclusion

The Trust has worked hard to develop a comprehensive, innovative, and unique M&E system that informs its strategies and management decision making. The system is well understood and has been implemented. The monthly, quarterly and annual monitoring systems function well. The impact assessment tools need more development – with low response rates to the baseline questionnaire affecting its reliability and a range of challenges which have limited the grant partner analysis report on ‘Knowing Civil Society’. These will be further improved this year.