Health Promotion International Advance Access published online on April 10, 2008
Health Promotion International, doi:10.1093/heapro/dan013
| ||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||
Article |
Building capacity for AIDS NGOs in southern Africa: evaluation of a pilot initiative
1 Yale School of Public Health, 60 College St., Yale University, New Haven, CT 06520-8034, USA 2Global Health Division, Yale School of Public Health, 60 College St., Suite 318, PO Box 208034, New Haven, CT 06520-8034, USA 3 Wits Institute for Social and Economic Research, University of Witwatersrand, South Africa
* Corresponding author. E-mail: kari.hartwig{at}yale.edu
 |
SUMMARY |
|---|
 TOP SUMMARY INTRODUCTIONCAPACITY BUILDING FOR NGOSTHE NGO INSTITUTEEVALUATION DESIGNRESULTSDISCUSSIONFUNDINGREFERENCES |
|---|
In this paper, we present the evaluation results of an AIDS non-governmental organization (NGO) capacity building 20-month pilot initiative in five countries in southern Africa called the NGO Institute. A five-person international team conducted a 2 week evaluation of the pilot in 2004 to assess the strength of the model, designed and funded by Bristol-Myers Squibb Foundation. The NGO Institute functioned through a separate consortium in each country. Results of the pilot indicate variations in adaptation and implementation of the model in each of the five countries. Each consortium took considerable time to develop its own governance and management systems. There were examples of strengthened NGO capacity in each country although it was too soon to establish overall impact. The strengths and weaknesses of this NGO capacity building model are presented along with the implications for other funding agencies and NGOs.
Key words: evaluation; capacity building; AIDs; NGOs
|
INTRODUCTION |
|---|
|
TOPSUMMARYINTRODUCTIONCAPACITY BUILDING FOR NGOSTHE NGO INSTITUTEEVALUATION DESIGNRESULTSDISCUSSIONFUNDINGREFERENCES |
|---|
The language of ‘capacity building’ has become ubiquitous among international funding agencies over the past 15 years, both in referring to governments and non-governmental organizations (NGO) (Eade, 1997
; Crisp et al., 2000; LaFond et al., 2002; Potter and Brough, 2004). In the health sector, as developing nations grapple with significant epidemics of AIDS, tuberculosis and malaria and a concomitant poor primary health care infrastructure, the newer language of ‘scaling up’ programs and services to meet these demands is commonly heard (Binswanger, 2000). As Potter and Brough (Potter and Brough, 2004) note, too often capacity building is equated with training, a one-time financial input, or short-term external technical assistance. The premise of capacity building—and its popularity—is the assumption that it will lead to institutions better able to address the health needs of their communities in both the short- and long-term (Hawe et al., 1997). However, too often we have seen that efforts to build organizational capacity do not guarantee effectiveness or sustainability of institutions or programs (Shediac-Rizkallah and Bone, 1998; Godfrey et al., 2002). In this paper, we present the evaluation results of an AIDS NGO capacity building pilot initiative. We begin with a definition of capacity building and an overview of various mechanisms used to develop capacity. Next we describe the program design, the assumptions concerning inputs and outputs and the achievements accomplished by the end of the 20-month pilot. Finally, we conclude with lessons from this pilot capacity building initiative and implications for funding agencies, NGOs and governments as we collectively work to strengthen the public and private sectors in AIDS prevention and care programs.
|
CAPACITY BUILDING FOR NGOS |
|---|
|
TOPSUMMARYINTRODUCTIONCAPACITY BUILDING FOR NGOSTHE NGO INSTITUTEEVALUATION DESIGNRESULTSDISCUSSIONFUNDINGREFERENCES |
|---|
There are numerous definitions for capacity building; we have identified one by Potter and Brough that captures the objectives of the pilot initiative we evaluated: ‘It [capacity building] should enable program execution independent of changes of personalities, technologies, social structures and resource crises, i.e. it implies developing sustainable, and robust, systems’ (p. 337). Crisp et al. (Crisp et al., 2000
) identified four commonly used approaches. These included a top-down organizational approach (e.g. policy); a bottom-up organizational approach (e.g. staff training); a partnership approach and a community organizing approach. Common mechanisms for developing organizational capacity comprise increasing the southern partner’s decision-making power; direct funding; management training through courses, visits with other NGOs; consultancy visits by northern or local consultants; workshops and conferences, among others (James, 1994). Most funding agencies adopt two or three of these strategies to achieve their capacity building objectives.
We could find only one example in the literature of different capacity building models for NGOs in developing countries working in HIV and AIDS (Kotellos et al., 1998) and one example from the USA (Napp et al., 2002). In this evaluation of the pilot NGO Institute capacity building initiative for AIDS NGOs in South Africa, Namibia, Botswana, Swaziland and Lesotho, we describe the model, its primary inputs and outputs and core lessons.
|
THE NGO INSTITUTE |
|---|
|
TOPSUMMARYINTRODUCTIONCAPACITY BUILDING FOR NGOSTHE NGO INSTITUTEEVALUATION DESIGNRESULTSDISCUSSIONFUNDINGREFERENCES |
|---|
In 1999, Bristol-Myers Squibb Foundation initiated the Secure the Future (STF) program in southern Africa (Botswana, Namibia, South Africa, Lesotho and Swaziland) as a 5-year, US$100 million program to mitigate the impact of HIV and AIDS on women and children in the region. The six-person STF staff met bi-annually with an external Technical Advisory Committee (TAC) made up of 13 experts from the region including academics, community leaders and government representatives to review community grants to NGOs on HIV prevention and AIDS care. Persistently poor quality in the NGO proposals prompted the TAC to recommend concerted efforts in NGO capacity building. Within 4 months, STF staff launched the pilot ‘NGO Institute’, and hired a full-time contractor to manage the project.
The 20-month pilot project set out to build organizational capacity in AIDS NGOs and community-based organizations (CBOs) specifically in the areas of management, leadership and governance. (We use CBO in the developing country context to mean organizations with nascent organizational structures, often made up of volunteers, with limited to no financial or organizational management systems in place.) This included financial and project management, human resource development, governance systems (e.g. boards of directors) and developing monitoring and evaluation (M&E) systems. The call for proposals required that a coalition of agencies come together in each country to implement a local ‘NGO Institute’. The budget amounts for each consortium varied, ranging from $200 000 to $300 000 per country for the full 20-month project (see Table 1 for a budget summary of each consortium). In addition, BMSF had contracts with other institutions to support report coordination, documentation, M&E, marketing and team building activities for the five consortia.
|
STF envisioned that each consortium would work through four phases: (i) start up, (ii) needs assessment of the NGO sector, (iii) development of training modules and finally, (iv) delivery of trainings and a mentoring program to NGOs and CBOs identified during the needs assessment. STF staff and TAC members recognized that no significant impact on the NGO sector could be achieved in the 20-month timeframe. They did, however, have a number of expected outputs to set the stage for longer term impact, including the production of modules for training in various aspects of organizational capacity development, the delivery of trainings and mentoring to NGOs and CBOs, and strengthened systems for future program delivery. In addition, it was expected that the NGO Institute in each country would document its process, market its courses and implement a consortium-wide M&E system. A conceptual framework of the pilot program with the project inputs and expected outputs is shown in Figure 1.
|
|
EVALUATION DESIGN |
|---|
|
TOPSUMMARYINTRODUCTIONCAPACITY BUILDING FOR NGOSTHE NGO INSTITUTEEVALUATION DESIGNRESULTSDISCUSSIONFUNDINGREFERENCES |
|---|
The primary purpose of the evaluation was to critically appraise the concept and the development process of the NGO Institute in each country, the value of the technical assistance and the primary outputs. A process evaluation of the NGO Institute had already been conducted. As a result, our primary evaluation questions centered on: what was the value of the model?
A five-person international team led the evaluation from 14 to 25 June 2004, dividing visits between the five countries. Team members came from Yale University, the University of Pretoria, a Namibian human rights NGO and also included a Civicus-South Africa Visiting Fellow from India. The team’s expertise included NGO organizational development and management in developing countries, particularly Africa, evaluation, program management and systems development, and training.
|
RESULTS |
|---|
|
TOPSUMMARYINTRODUCTIONCAPACITY BUILDING FOR NGOSTHE NGO INSTITUTEEVALUATION DESIGNRESULTSDISCUSSIONFUNDINGREFERENCES |
|---|
Project start-up: consortium formation
Each country team worked to develop its consortium identity during the first months. In some of the countries, such as Namibia, a self-selected group of four agencies came together to form the local consortium. In other countries, a nexus of collaborative agencies formed on their own but in the process of coalition formation, BMSF required them to include additional specific agencies. These coalitions were euphemistically referred to as ‘forced or arranged marriages’ by the consortium members and for some, this had negative consequences.
The most problematic underlying assumption in the program theory that became apparent during the evaluation was underestimating the time required to establish (i) a consortium identity, (ii) systems and procedures for working together and (iii) trust that each partner would play its role and had the skills necessary to do so. Coalition partners represented different constituencies, geographies and skills. Partners included a variety of organizational types, such as private agencies, training institutes, universities, NGO umbrella organizations and consulting groups. Each had to become accustomed to the different organizational cultures of the others. The consortia had to determine shared budget, workplans, resources and decision-making roles collectively in a process few had experienced before.
Two coalitions quickly ran into difficulties. In South Africa, a clash of organizational cultures and racial tensions split the coalition of four into two pairs of partnerships after the initial 6 months. Two Afrikaner agencies partnered to work in one rural region; in a neighboring rural province the other two agencies, an international African research group and a local School of Public Health worked together. Having successfully split the geographic responsibilities between them, the two partnerships functioned well throughout the rest of the pilot period. In Botswana, a coalition of four NGOs and one training institute ran into issues of trust and role definition. The lead agency eventually dropped out of the coalition and was replaced by another NGO. The other consortia had no unusual challenges in working together.
During the evaluation, the consortium participants made the following observations about their own assumptions about working as a consortium:
- ‘Working together as a consortium would be easy and smooth’.
- ‘Once we committed ourselves to working as a consortium, I thought we would be a consortium; instead, everyone came to the table wearing their institutional ‘hats’ or identities’.
- ‘Given the high prevalence of AIDS in our country [40%], I thought all consortium members would see this project as an emergency and high priority’.
- ‘We assumed that organizations in the consortium had the appropriate skills and knowledge to deliver the program’. (Evaluation field notes, South Africa, 6 August 2004.)
|
Developing coalition systems and external inputs
As a pilot, there was a scope for each consortium to experiment with different governance structures. All consortiums had a working committee, which included the program coordinator from the lead organization or secretariat and focal persons from each partner. In addition, all but the South Africa partnerships had an oversight structure that included management from each of the partners, usually the directors and sometimes ministry of health and TAC representatives as well. The frequency of these meetings varied. Most of the work happened at the level of the program coordinators and these groups tended to meet more often. However, the sign-off of each director required to move forward on major decisions at times acted as a barrier. The greater frequency of meetings and clear consortium decision-making roles assisted Namibia and South Africa in moving forward more rapidly through various phases of the pilot than some of the other consortia. For example, in Swaziland, the consortium held no meetings or activities during 5 months when the project director of the lead agency was out of the country.
BMSF contracted with a local subcontractor to design a common M&E system to capture project process and outputs and further enhance coalition systems and infrastructure development. The subcontractors created an electronic database system that would generate compatible data across countries as well as have the adaptability to create unique indicators for each consortium as appropriate. The lead agency in each consortium was to maintain the M&E system with each partner providing data monthly into the system. The M&E plan was intended to fit the management and monitoring needs of both the consortium agencies and the donor, but in practice a number of weak links emerged. The electronic database system was a product the subcontractor delivered to BMSF—a well designed system with reasonable indicators and a system compatible with the funders’ computer database systems. However, the electronic database system did not work with the existing computers of four of the six partnerships and BMSF expected purchases for new computers to come from agencies’ existing funds not the project grant. The two lead agencies which found the M&E system favorable both remarked that they were able to utilize and adapt the system for other projects and that it helped them in applying for other grants.
The 2 year project consultant hired by BMSF to manage and guide the NGO Institute provided an intangible benefit to all consortia members. As a member of the TAC, she had advocated for BMSF assistance in NGO capacity building and then accepted the offer to put her words into action by BMSF. She played multiple roles as mentor, advocate, mediator, disciplinarian, negotiator and innovator. In her role as project champion, she pushed each consortium to achieve its best results. A few consortium members found her technical assistance ill-timed or that she showed favoritism among agencies, but overall most felt she provided critical assistance for the consortia in coalition formation, maintenance and moving towards their objectives.
Needs assessments, module development, trainings and mentoring
Although there was a clear timeline established by the donor (see Table 2), the timing of each consortium in achieving different phases of the pilot varied. During the evaluation, a number of consortium members stated that they felt the progress of each consortium was made to feel like a competition between countries during the two regional review meetings of the NGO Institute members. In this ‘race,’ the Namibia and South African consortia were consistently held up as the frontrunners, moving into the training and mentoring phases by the time of the midterm evaluation in late 2003, while Botswana, Swaziland and Lesotho did not initiate training until shortly before the final evaluation in June 2004. Many individual factors contributed to the rate of implementation in the different countries. For example, the Botswana consortium placed a significant emphasis on building a team, with large community gatherings. Other partnerships, including the two South African partnerships, placed a priority on meeting the objectives set out by BMSF in a timely fashion.
Conducting the NGO/CBO needs assessment in each country (part of Phase 2) was an important first step in determining who the audience would be for the trainings, and their particular training needs. In a number of countries, consortium members used a detailed self-assessment survey that was provided to NGO managers to identify strengths and gaps in management systems and occasionally in HIV and AIDS technical programming skills. With the exception of South Africa and Botswana, all of the NGO assessments concentrated geographically on NGOs or CBOs in and around the capital city. In South Africa when BMSF staff consulted with Ministry of Health officials about the NGO Institute initiative, they designated two rural, under-resourced provinces as the target. Both of the South African partnership teams were challenged to reach and identify CBOs 500–600 km from their primary office base. In Botswana, three regional gatherings were held with potential stakeholders, one in the capital region and two in other areas of the country.
The conceptualization of the NGO Institute sought a strong model that could be used in all five countries. Developing curriculum appropriate for NGO capacity building was one key aspect of the formation of the project. No two consortia developed the same set of training modules. In part, this represented the existing strengths of the training institutions and universities which already had existing curriculum that could be easily adapted. All five countries developed modules on leadership and financial management as seen in Table 3, but from there the topics varied widely from governance, resource mobilization and human resources management to HIV and AIDS in the workplace. Some of the strengths of the modules included clear learning objectives and comprehensive material, including variation of models and theories in the governance and management modules. The weaknesses included a lack of participatory methods to transform information and data to skills development and lack of audience identification and sensitivity to gender issues.
|
Each country provided training in two or more of the modules for 40 to 60 NGOs and CBOs (Phase 4). Collectively, the consortiums trained more than 1700 individuals across the five countries. The process evaluation conducted by an external subcontractor found that participants indicated that the overall quality of the trainings were good. Weaknesses identified in the training delivery included a lack of understanding of the NGO/CBO sector and culture by some of the university and business sector trainers. Further, several of these trainers were unfamiliar with some of the unique issues pertinent to HIV and AIDS prevention and care programs and how those might affect the management operations of the NGOs. Examples included staff with relatively limited formal education but reflecting the populations affected by HIV; people living with AIDS who may have multiple sick leaves reducing consistent staff coverage; difficult logistical settings for program delivery; the emotional toll of care and support programs; and how HIV stigma affected the work of the staff and agency. The NGO Institute consortia members also identified other challenges: attendance by inappropriate staff unable to utilize the trainings, inconsistent attendance by trainees and attendance by former NGO managers whose NGOs had dissolved.
All consortia provided some degree of mentoring to the NGOs after training but the most intensive mentoring was provided by the South African and Namibian consortia. Two consortia provided training through modules; others designated specific agencies to do weekly follow ups with NGOs to facilitate transfer and adaptation of the new skills. For some of the rural CBOs, mentors were placed in the rural townships and villages for weeks at a time to help facilitate skills transfers.
At the end of the 20-month pilot, the NGO Institute had moved through all phases of the project in each country. Each consortium had established operational systems and had a nascent sense of shared identity and purpose. Although no long-term impact was expected, in each country there were examples of NGOs or CBOs who had strengthened their internal management and governance structures as well as others which had written proposals to external funders such as the Global Fund and received funding support. At the report and discussion of the evaluation findings in August 2005, each consortium team developed a business plan for continuing their individual consortium.
|
DISCUSSION |
|---|
|
TOPSUMMARYINTRODUCTIONCAPACITY BUILDING FOR NGOSTHE NGO INSTITUTEEVALUATION DESIGNRESULTSDISCUSSIONFUNDINGREFERENCES |
|---|
The NGO Institute pilot represented an ambitious 20-month initiative to create a model of AIDS NGO capacity building in five countries in southern Africa. The pilot demonstrated two levels of organizational capacity building. The first level was the capacity building required for each NGO Institute consortium group to develop its own governance system, operations and materials. The second level was the provision of training and mentoring to the AIDS NGOs to strengthen their own operations and ability to deliver programs.
The formation of these coalitions to deliver organizational capacity building for AIDS NGOs and CBOs in these countries represented an innovation. The structure focused on three aspects of organizational capacity building for AIDS NGOs defined by Kotellos et al. (Kotellos et al., 1998), namely the management and financial skills to run the organization, financial sustainability to continue to provide services and the political capacity to maintain the support of the community and other key stakeholders. Ironically, the consortia appeared more successful at transferring these skills and abilities than assuring their own financial sustainability to continue on their own at the end of the pilot without BMSF funding. This result was likely a combination of factors: (i) an expectation that BMSF would fund the roll-out of the Institute; (ii) a focus on project deliverables such as the number of NGOs trained and mentored; (iii) limited time to prepare applications for alternative funding grants as each agency partner also had other projects it was implementing in addition to the pilot; and (iv) a fragile consortium identity with mixed levels of ownership.
A further unique feature of the NGO Institute was the characteristic that all of the consortia members, BMSF staff and subcontractors were from southern Africa, representing the rich, multi-ethnic, multi-lingual diversity of the region. Thus, there was no north–south tension in the implementation and no historical memory to explain. There was a sense of pride among participants that this was an ‘African model’ of sorts; although with it came the indigenous political tensions as one would expect in any region. This pride was a unique strength of the model. In addition, both organizations and individuals developed important experience in understanding the skills and inputs required to successfully operationalize a consortium.
The component of mentoring incorporated in the NGO Institute design is another unique feature and strength of the model. The operationalization of this component was best demonstrated by the South African and Namibian consortia which conducted targeted follow up with NGOs and CBOs post training in order to assist the organizations to incorporate changes. This additional component of understanding organizational learning is one that few organizational capacity building training programs are able to address.
The weaknesses of the model in delivering training and mentoring in organizational capacity building were ones that have been commonly seen in similar efforts. These included insufficient time for organizational development and capacity building, overly optimistic expectations of rapid change, having people attend the trainings who are not appropriate for the subject matter, trainees attendance being inconsistent, an uneven mix of NGO and CBO participants with uneven skill needs and occasionally trainers from the professional world or university who were unfamiliar with NGO culture and staff needs and so did not adapt the training appropriately. As a 20-month pilot, it would be unrealistic to expect more from this initiative than the activities accomplished. Future initiatives that use a consortium model should allocate additional time to support the development of the governance structures for the consortium itself. A further lesson is to caution funding agencies in creating ‘forced marriages’ of organizations into a consortium. Although funding agencies often know a broader network of organizations than many NGOs and CBOs and can play a critical role in introducing agencies to one another with shared or complementary missions, forcing them into a partnership is likely to produce resentment.
Within a year of the conclusion of the pilot initiative BMSF ‘rolled-out’ a 3 year NGO Institute with a revised model that did not include the consortium structure in each country but did include the same core objectives. The roll out plan also incorporated many of the curriculum modules developed under the pilot.
There were a number of successes and failures in this NGO capacity building model. The attention to AIDS NGOs and CBOs reflects the proliferation of these organizations in the region, and their often nascent skills in management, financing and governance. Strengthening institutions in this sector to be effective and sustainable can only bolster the overall response to the epidemic. A cautionary note to funders, however, is that they need to work with local institutions which share the values for long-term sustainable organizational change rather than those who are looking for the next grant to maintain their organization. We need more descriptions and evaluations of different approaches to strengthening organizational capacity building for AIDS NGOs in order to learn and improve future capacity building initiatives.
|
FUNDING |
|---|
|
TOPSUMMARYINTRODUCTIONCAPACITY BUILDING FOR NGOSTHE NGO INSTITUTEEVALUATION DESIGNRESULTSDISCUSSIONFUNDINGREFERENCES |
|---|
Funding provided by Bristol-Myers Squibb Foundation.
|
REFERENCES |
|---|
|
TOPSUMMARYINTRODUCTIONCAPACITY BUILDING FOR NGOSTHE NGO INSTITUTEEVALUATION DESIGNRESULTSDISCUSSIONFUNDINGREFERENCES |
|---|
Binswanger H. P. Scaling up HIV/AIDS programs to national coverage. Science (2000) 288:2173–2177.
Crisp B. R., Swerissen H., Duckett S. J. Four approaches to capacity building in health: consequences for measurement and accountability. Health Promotion International (2000) 15:99–107.
Eade D. Capacity Building: An Approach to People-Centred Development (1997) Oxford: Oxfam.
Godfrey M., Sophal C., Kato T., Piseth L. V., Dorina P., Saravy T. Technical assistance and capacity development in an aid-dependent economy: the experience of Cambodia. World Development (2002) 30:355–373.[CrossRef][Web of Science]
Hawe P., Noort M., King L., Jordens C. Multiplying health gains: the critical role of capacity-building within health promotion programs. Health Policy (1997) 39:29–42.[CrossRef][Web of Science][Medline]
James R. Strengthening the Capacity of Southern NGO Partners (1994) Oxford: INTRAC.
Kotellos K. A., Amon J. J., Benazerga W. M. G. Field experiences: measuring capacity building efforts in HIV/AIDS prevention programmes. AIDS (1998) 12:S109–S117.[Web of Science]
Lafond A. K., Brown L., Macintyre K. Mapping capacity in the health sector a conceptual framework. International Journal of Health Planning and Management (2002) 17:3–22.[CrossRef][Web of Science][Medline]
Napp D., Gibbs D., Jolly D., Westover B., Uhl G. Evaluation barriers and facilitators among community-based HIV prevention programs. AIDS Education and Prevention (2002) 14:38–48.[CrossRef][Web of Science][Medline]
Potter C., Brough R. Systemic capacity building: a hierarchy of needs. Health Policy and Planning (2004) 19:336–345.
Shediac-Rizkallah M. C., Bone L. R. Planning for the sustainability of community-based health programs: conceptual frameworks and future directions for research, practice and policy. Health Education and Research (1998) 13:87–108.
CiteULike 
Connotea 
Del.icio.us What's this?
| ||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||