Southern Africa Trust
Monitoring & Evaluation Baseline Survey
Report
April 2007
Produced for the Southern Africa Trust by NB Ideas, PO Box 12364 , Mill Street ,8010, South Africa , Email: info@nbideas.co.za
1
Introduction
The Southern
Africa Trust commissioned a baseline report to provide a quick snap shot of the
civil society sector and its understanding of regional policy processes. The
information gathered will be used to determine the nature of capacity building,
funding and networking supported by the Trust.
In addition,
the information provides a baseline against which the Southern Africa Trust
will measure the impact of its work by repeating the survey in a few years
time.
Following this introduction, the report
outlines the process and methodology used in Section Two. In Section Three an
overview of the respondents is provided. This is followed in Section Four, with
a summary of the findings grouped into five categories, namely; capacity,
evidence based research, networking, knowledge and participation. The findings
are analysed with reference to area of operation, country of location, and type
of organisation. The report concludes with general comments and lessons for
future surveys.
2
Process and methodology
The questionnaire used was developed by the
consultants with inputs from Southern Africa Trust staff. The draft
questionnaire was piloted with the team before being rolled out. The
questionnaire aims to provide baseline information in respect of capacity of
civil society organisations to engage in regional policy processes to end
poverty. This is assessed using two indicators: human resources and funding.
Secondly, the questionnaire provides information on the extent and nature of
evidence-based research taking place. Thirdly, questions on the nature, extent
and frequency of networking provide an overview of collaboration efforts. The
next set of questions tried to determine the knowledge base of partners through
a series of multiple choice questions. The final set of questions focused on
the degree to which organisations are participating in policy processes and the
obstacles to increased participation.
The questionnaire was administered in
English, French and Portuguese. In January (17/01) emails requests were sent
out to selected organisations from the
Southern
Africa Trust
database, a request was placed on the Trust
website, forms were printed and two people were asked to administer the forms
at the World Social Forum in Nairobi in January 2007.
A second email was sent out a week before
the closing date (02/02) reminding people to complete the questionnaire. This
was followed a week later with an email to those who had not completed the form
announcing a one-week extension and making a final appeal.
On analysing the responses received it
became clear that the Trust had not received adequate responses from some
countries. Expanded databases were compiled for
Angola
,
Namibia
and
Botswana
and a new request circulated to these organisations. In the case of the
Democratic Republic of the Congo (DRC), contact was made with a partner
organisation requesting them to assist in identifying additional contacts and
circulating the questionnaire.
The dataset was finally closed at the end
of March 2007. The final set of
responses reflects the strength of regional policy organisations in the
Southern African Development Community (SADC) region. A reasonable number of
organisations responded from
South Africa
,
Zambia
,
Tanzania
and
Mozambique
. Fair responses were
received from
Botswana
,
Zimbabwe
and
Malawi
. Responses from other
countries were limited. Consequently, while some general conclusions about the
sector are possible, country by country analysis is limited to major trends
given the limited sample size.
As respondents were able to select how they
describe the work of their organisation the survey merely provides a perception
of the sector and its work by the sector itself. No changes were made to
completed questionnaires. The authors of the report did have concerns about the
accuracy of responses, and this is reflected in the text describing the
analysis of the dataset.
3
Respondents
The database provided by the Trust was
limited with only 273 relevant organisations captured on the initial dataset. A
further 132 organisations were included in the subsequent databases added.
From a database of 405 organisations, 125
(31%) survey responses were received. Twenty-one
respondents used the online form, 104 (83%) used the Microsoft Word form or
printed form. Of the 125 respondents 70 (56%) did not fully complete the survey
leaving one or more question unanswered. Five of the responses were excluded as
the organisations in question operated beyond SADC, the focus of the Trust’s
interventions.
3.1
Analysis of respondents by geographic scope of operations
The bulk of respondents work focused on a
country (54). This was followed by organisations operating at a regional scale,
(32) those focusing on the African continent (18) and finally those operating
globally (20) and one (1) undefined.
Graph
1: Analysis of respondents by scope of operation
The largest slice of the pie above is for
organisations with a single country focus. In the table below we list the
number of respondents per SADC country and the number of these focusing just on
their country of location.
The largest number of responses were
received by
South Africa
(29),
followed by
Zambia
(19),
Tanzania
(13) and
Mozambique
(11). This probably
reflects, in part, the Trust’s profile in these countries and in part the
strength of CSOs and therefore the number of organisations on the
database. The fewest responses were
received from
Lesotho
(1),
Namibia
(1), The Democratic Republic of Congo (1),
Madagascar
(2) and
Swaziland
(3). The database for the DRC,
Lesotho
and
Madagascar
was relatively small. Of concern is the poor response from Namibian organisations
as the database for
Namibia
was extensive.
Within the other category 15 respondents
did not specify either the name of the organisation or the country of operation
so it was not possible to classify them. A further 3 were based outside of SADC
but specified they worked in the SADC region. Any organisations who worked
outside of the SADC region were excluded from the final dataset.
Table
1: Analysis of respondents by country of location
Country
|
No.
of respondents by country
|
No.
of respondents working only in their own country
|
Angola
|
4
|
3
|
Botswana
|
6
|
0
|
Democratic
Republic
of
Congo
(DRC)
|
1
|
1
|
Lesotho
|
1
|
1
|
Madagascar
|
2
|
2
|
Malawi
|
5
|
5
|
Mauritius
|
4
|
2
|
Mozambique
|
11
|
9
|
Namibia
|
1
|
1
|
South Africa
|
31
|
3
|
Swaziland
|
3
|
0
|
Tanzania
|
13
|
10
|
Zambia
|
17
|
7
|
Zimbabwe
|
7
|
5
|
Other
|
18
|
5
|
TOTAL
|
125
|
54
|
3.2
Analysis of respondents by type of work
A second lens used to analyse the responses
is the type of work performed by an organisation, namely; advocacy, research, service
delivery, community development, policy or some combination of these. The
majority of organisations who responded to the questionnaire were involved in a
combination of research, policy, advocacy and development work (69). This was
followed by a focus on community development (22), research (7) and advocacy
(8). Eleven organisations focused on categories outside the options such as
media and one respondent did not state the type of work undertaken.
Disaggregating those engaged in a
combination of activities and including them in each of the types of work they
do, the respondent profile is fairly evenly balanced with service delivery the
least represented. The graph below provides a visual picture of the responses
by type of work.
Graph
2: Analysis of respondents by type of work
4 Findings
4.1
Capacity
In general,
the capacity of organisations who responded to the questionnaire is divided
with just under half considered to be adequately resourced and just over half
under resourced.
If human resource
strength is used as the measure of capacity 32, or one quarter, of the
organisations would be considered highly capacitated in respect of policy work.
The effectiveness of these organisations has not been evaluated. However all 32
reported staff of 7 or more working in the policy arena. A further 3
organisations did not have significant staff but did have a large pool of
consultants contracted to do the policy work on their behalf. Twenty-five
organisations (21%) were adequately resourced largely with internal staff but
two with consultants. These organisations had between 4 and 7 staff. The
remaining 53% did not have adequate capacity with 19% reporting no capacity and
34% fewer than four staff or consultants.
In general
respondents noted a high level of capacity, which does not match with the
team’s knowledge of the sector. It is doubtful where the reported human
resource capacity reflects full-time dedicated staffing and is more likely to
reflect a portion of staff whose activities includes some policy work. In the
view of the authors of this report this highlights the lack of understanding
about analytical policy work and its importance within the sector.
Graph 3: Consolidated analysis of human
resource capacity profile of respondents
If funding
is used as an indicator of capacity then the percentage of capacitated
organisations reduces substantially with only 1% regarding themselves as adequately resourced and a further 8% considered to be resourced by the
consultants, with a combination of dedicated resources for policy work and more
than three funding sources for this work. In aggregate terms roughly half (54%)
do not have adequate funding resources suggesting a direct link between funding
and human resource capacity. In addition to funding resources, 16% of
respondents raised funds through consulting services, 8% received funding from
government (within this 40% were in Tanzania and 30% in South Africa) and 16%
raised funds directly through events and membership fees. Twelve organisations
worked with volunteers.
Graph 4: Consolidated analysis of funding capacity profile of
respondents
Where
capacity is weak or poor there is a strong correlation between human resource
and funding capacity. As organisations strengthen there is a window period where
human resource capacity exceeds the dedicated funding resources before the
balance swings and funding exceeds human capacity. This seems to suggest that strongly
capacitated organisations do not need additional funding support.
If the
capacity needs are further broken down, it is clear that the pool of
capacitated organisations is weighted toward organisations that are global or
continent focused with those working in individual countries (
Mozambique
,
Botswana
.
Zimbabwe
and
Tanzania
) recording the lowest
capacity, followed by those focusing on the SADC region. Angolan, Botswanan,
Zambian and South African CSOs recorded the strongest human capacity. Sixty-one
percent of organisations focusing on
Africa
have adequate to strong capacity. Fifty-five percent of organisations working
at the global level have adequate to strong capacity. In both cases, the
consolidated picture for human resource and for funding capacity is the same.
Organisations focusing on SADC had the poorest human resource capacity while
those focusing on a single country had the weakest funding capacity.
Table 2: Capacity assessment by scope of operations (human resources)
Scope
of operations
|
Weak
|
Poor
|
Adequate
|
Strong
|
Global
|
15%
|
30%
|
25%
|
30%
|
Africa
|
22%
|
17%
|
11%
|
50%
|
SADC
|
13%
|
53%
|
16%
|
19%
|
Country
specific
|
24%
|
31%
|
24%
|
20%
|
Total
|
19%
|
34%
|
21%
|
26%
|
Table 3: Capacity assessment by scope of operations (funding)
Scope
of operations
|
Weak
|
Poor
|
Adequate
|
Strong
|
Global
|
25%
|
20%
|
5%
|
50%
|
Africa
|
17%
|
22%
|
0%
|
61%
|
SADC
|
6%
|
44%
|
9%
|
41%
|
Country
specific
|
31%
|
33%
|
15%
|
20%
|
Total
|
22%
|
32%
|
10%
|
36%
|
Analysed by
type of work undertaken by the organisation, the strongest human resource
capacity exists in organisations involved in service delivery and/or research
and the weakest in organisations focusing on community development and/ or
other work, which (in most instances) was described as media work.
Table
4: Capacity assessment by type of organisation (human resources)
Type
of organisation
|
Weak
|
Poor
|
Adequate
|
Strong
|
Advocacy
|
16%
|
33%
|
29%
|
22%
|
Service
Delivery
|
13%
|
26%
|
35%
|
26%
|
