Health Promotion International, Vol. 16, No. 4, 367-380,
December 2001
© Oxford University Press 2001
Setting standards in the evaluation of community-based health promotion programmes— a unifying approach
Territory Health Services, Darwin, and Deakin University, Australia and 1 Institute of Health Promotion Research, University of British Columbia, Vancouver, Canada
Address for correspondence: Ms J. Judd Territory Health Services PO Box 40596 Casuarina 0812 NT Australia
 |
SUMMARY |
|---|
 TOP SUMMARY INTRODUCTIONVALUES AND RELATED ISSUES...OBJECTS OF INTEREST IN...THE SPIRIT OF THE...THE USE OF STANDARDS...MOVING TOWARD...ACKNOWLEDGEMENTSREFERENCES |
|---|
Community-based health promotion often emphasizes elements of empowerment, participation, multidisciplinary collaboration, capacity building, equity and sustainable development. Such an emphasis may be viewed as being in opposition to equally powerful notions of evidence-based decision making and accountability, and with funders’ and government decision-makers’ preoccupation with measuring outcomes. These tensions may be fuelled when community practitioners and lay participants feel evaluations are imposed upon them in a manner that fails to appreciate the uniqueness of their community, its programme, and practitioners’ skills and experience. This paper attempts to provide an approach that depicts evaluation as being mutually beneficial to both funders/government and practitioners. First, a values stance for health promotion, termed a ‘salutogenic’ orientation, is proposed as a foundation for the evaluation of community-based health promotion. Secondly, we discuss possible objects of interest, the first component of an evaluation. We then discuss the spirit of the times and its implications for community-based health promotion. Finally, we address the key question of setting standards. A typology of standards is presented. Arbitrary, experiential and utility standards are based on perceived needs and priorities of practitioners, lay participants or professional decision-makers. Historical, scientific and normative standards are driven by empirical, objective data. Propriety and feasibility standards are those wherein the primary concern is for consideration of resources, policies, legislation and administrative factors. The ‘model’ standards approach is presented as an exemplar of a combined approach that incorporates elements of each of the other standards. We argue that the ‘optimal’ standard for community-based health promotion depends on the setting and the circumstances. There is no ‘magic bullet’, ‘one-size-fits-all’ or ‘best’ standard. Further, we argue that standards should be set from an inclusive, salutogenic orientation. This approach offers a means of creating a situation in which policy-makers and funders are more supportive of evaluation designs that fit with community realities, and community stakeholders are more capable and consistent in rigorously evaluating community-based health promotion programmes and policies.
Key words: community-based health promotion; evaluation; salutogenic; standards
|
INTRODUCTION |
|---|
|
TOPSUMMARYINTRODUCTIONVALUES AND RELATED ISSUES...OBJECTS OF INTEREST IN...THE SPIRIT OF THE...THE USE OF STANDARDS...MOVING TOWARD...ACKNOWLEDGEMENTSREFERENCES |
|---|
Community-based health promotion programmes often emphasize empowerment, participation, social and sustainable development, multidisciplinary collaboration, capacity building and equity. Empowerment, in its most general sense, refers to the ability of people to gain understanding and control over personal, social, economic and political forces in order to take action to improve their life situations (Israel et al., 1994
), and capacity building refers to the problem-solving capability among individuals, organizations, neighbourhoods and communities (Hawe, 1994); in the context of health promotion workers, it refers to their ability to enhance the capacity of a system to prolong and multiply health effects, which represents a ‘value added’ dimension to health outcomes offered by any particular health promotion programme (Hawe et al., 1998). This emphasis is often juxtaposed with equally powerful notions of evidence-based decision making and accountability, in that funders and government decision-makers are frequently more concerned with measuring outcomes and defining success. Community practitioners and lay participants often feel that evaluations are imposed upon them, and that the evaluation process does not appreciate the uniqueness of their community, its programme, and its resources and skills (Labonte and Robertson, 1996; Trussler and Marchand, 1998). Portraying these viewpoints in a dichotomized manner may appear overly simplistic. We present them in this manner to assert that the issues raised by both sides represent legitimate concerns within the practice of community-based health promotion. It is essential to recognize that all parties involved are seeking to provide the most worthwhile programmes or policies to a designated community. The progression from programme objectives, to strategies employed, to data collected, to definitions of programme success is often not operationally articulated in a transparent, measurable fashion. The focus of this paper is on the latter stage (i.e. setting of standards), which we argue is the least developed in health promotion.
While concerns for accountability and outcomes are part of our current zeitgeist (spirit of the times), evaluation should not be a disempowering process. Rather, it can contribute to ‘the process of enabling people to increase control over, and to improve their health’ (World Health Organization, 1986). If practitioners are provided with adequate support for conducting an evaluation, they are highly motivated in knowing if they are making a difference, and how they can improve their programme.
This paper provides an approach that depicts evaluation as being mutually beneficial to all stakeholders. Our aim is to make the evaluation process more transparent and collaborative so that all parties will be satisfied, and gain from the outcomes of community-based health promotion evaluations. It will assist practitioners and decision-makers in defining programme success at the outset of a programme and/or its evaluation, rather than being the last issue discussed. The issues raised in this paper are closely related to the sustainability and improvement of programmes, and the health of the communities served. Practitioners are often concerned that their programmes will not be continued due to a perceived lack of success by decision makers.
We assert that standards for the evaluation of community-based health promotion are, for the most part, implicitly defined or assumed. Secondly, we assert that standards employed in the evaluation of community-based health promotion that are not expressed may succumb to the same fate as health promotion indicators; i.e. they ‘are often predetermined and shaped by those in political, administrative and economic fields’ [(St Leger, 1999), p. 194]. More importantly, they are often ignored or forgotten as a programme proceeds over time. We endorse making the use of such standards more explicit and transparent in a collaborative process. We recognize that the word ‘standard’ may have a pejorative connotation for some stakeholders in community-based health promotion; however, we argue that all stakeholders consciously or unconsciously employ their own standards in assessing programme quality.
The first section of the paper underscores the values and philosophy of health promotion as it relates to the evaluation process. Next, we discuss possible objects of interest, the first component of an evaluation, in community-based health promotion. Then, we further explicate the spirit of the times and its implications for community-based health promotion. Finally, we present a taxonomy of ‘standards’ for evaluating community-based health promotion against which such objects of interest can be measured.
For our present purposes, we endorse the definition of evaluation proposed by Green and Kreuter (Green and Kreuter, 1999), namely that evaluation involves the comparison of an object of interest against a standard of acceptability. We believe all stakeholders have a role in articulating the objects of interest (e.g. changes in health status, community development, intersectoral collaboration) and standards of acceptability for a given programme or policy, not just the person who commissions the evaluation. We endorse the use of a comprehensive, diverse set of standards that reflects different concerns and forms of evidence related to the evaluation of health promotion programmes. This approach offers a means of creating a situation in which policy-makers and funders can be more supportive of evaluation designs that fit with community realities, and community stakeholders can become more capable and consistent in evaluating their health promotion programmes and policies. Evaluation should facilitate understanding by all stakeholders. Although we refer to community-based health promotion, the issues raised may be relevant to health promotion interventions at other levels (i.e. national) in other settings (i.e. workplace- or school-based health promotion) and other disciplines.
|
VALUES AND RELATED ISSUES IN THE EVALUATION OF COMMUNITY-BASED HEALTH PROMOTION |
|---|
|
TOPSUMMARYINTRODUCTIONVALUES AND RELATED ISSUES...OBJECTS OF INTEREST IN...THE SPIRIT OF THE...THE USE OF STANDARDS...MOVING TOWARD...ACKNOWLEDGEMENTSREFERENCES |
|---|
Community-based health promotion is explicitly concerned with a vision of a preferred future (Labonte, 1996
). This vision includes a viable natural environment, a sustainable economic environment, a sufficient economy, an equitable social environment, a convivial community and a liveable environment (Labonte, 1993). This ‘vision’ can be encapsulated in what Antonovsky termed as a ‘salutogenic’ orientation to health (Antonovsky, 1979; Antonovsky, 1996). We propose salutogenesis as the core or foundational value underlying the development, articulation and implementation of standards for community-based health promotion programmes or policies.
The word salutogenic derives from a combination of ‘salus’ meaning health, and ‘genesis’ meaning to give birth. Salutogenesis literally means ‘that which gives birth to health’. In traditional public health and community medicine approaches, a ‘pathogenic’ perspective, in which the focus is on disease or illness and its prevention or treatment, most often dominates interventions. Adoption of a salutogenic perspective highlights the importance of starting from a consideration of how health is created and maintained through community-based health promotion (Cowley and Billings, 1999). Salutogenesis suggests a link to notions of ‘social capital’, capacity building and citizen engagement in that it focuses on activities that seek to maximize the health and quality-of-life of individuals, families and communities. Social capital is defined here as:
‘the factor that allows collective action in the public sphere and for the common good. It is social cohesion, and comprises attention, engagement and trust of both non-familiar people and the institutions of governance. Social capital can be used to measure the capacity of the social linkages and their resilience or fragility. Social capital is iterative and experientially developed, and requires both levels of trust and competence in social interaction’ [(Cox, 1997
Using this approach, evaluation standards should maximize human health, quality-of-life and well-being. This view also recognizes that health has an instrumental value rather than being an end in itself.
We propose that, from a salutogenic orientation, evaluation standards in community-based health promotion must consider the values and pertinent issues of health promotion in appraising the success or failure of a given initiative (Labonte, 1996; Hancock et al., 1998). While the purpose of this paper is not to reiterate all such values and issues, we highlight those that we deem to be critical.
First, power is central to practice; accordingly, the proposed salutogenic view is consistent with an egalitarian approach that rejects professional dominance in the decision making surrounding programme evaluations. Programmes are evaluated ‘by real people in complex organizations that are marked by historically developed and structurally organized power relations and human wants and interests’ [(Cervero and Wilson, 1994), p. 249]. Multiple stakeholders (i.e. consumers/ citizens, practitioners, managers and decision-makers) have a role to play in evaluation. Health promotion demands coordinated action and collaboration among governments, health, social and economic sectors, non-governmental and voluntary organizations, local authorities, industry and the media to promote individual and community health. Scientific and local indigenous knowledge each have a contribution to make towards the evaluation of programmes and policies.
Secondly, the salutogenic view recognizes that health promotion is people-centred and collectivist (Raeburn and Rootman, 1998). It is at odds with a strong emphasis on individual responsibility for health that ignores the impact of social, cultural, economic and environmental determinants of health. Health promotion seeks to maximize the inclusion or involvement of individuals or groups who have been historically marginalized, such as Aboriginal peoples and the poor—those with the poorest health status. Participation and ownership of the programme by the community facilitates problem solving, builds community competence, and creates successful, sustainable programmes, rather than programmes that are imposed by outsiders (Eisen, 1994; Camiletti, 1996). Stakeholders of programmes and evaluations should recognize that communities are dynamic, and socially, culturally and economically heterogeneous. Therefore, strategies need to be adapted to local needs and possibilities.
Thirdly, the proposed salutogenic approach to community-based health promotion clearly questions the dominance of economic rationalism and market ideology in public policy (Labonte, 1996). It is explicitly concerned with the aforementioned vision of a preferred future that includes a viable natural environment and a sustainable economic environment. Its emphasis is on equity rather than productivity; and health rather than wealth.
Underpinning each of these values and the adoption of a salutogenic approach to setting standards is the notion of empowerment—increasingly recognized as a key element in the evaluation of community-based health promotion (Fetterman et al., 1996). Empowerment is usually described as a process, but may be considered an outcome variable (i.e. an object of interest) when capacity building is a major activity of a community intervention. Empowerment encompasses participation, multidisciplinary collaboration, equity, capacity building, and social and sustainable development (Hawe, 1994). Approaches such as empowerment evaluation (Fetterman et al., 1996), participatory research (Green et al., 1995), participatory evaluation (Health Canada, 1996) and ‘responsive constructive evaluation’ (Guba and Lincoln, 1989) foster the systematic generation of new knowledge and social capital. This is done through a process that builds upon the skills and experiences of all parties involved, and contributes to quality-of-life and well-being. Such evaluations move toward salutogenesis in that they offer viable possibilities for the evaluation of community-based health promotion programmes. They are commensurate with the unifying approach to setting standards as detailed in this paper.
With these principles in mind, evaluation is necessarily a collaborative group activity, fundamentally democratic, participatory, and must examine issues of concern to the community in an open forum. Evaluations of community-based health promotion programmes limited to aggregates of changes in health behaviour or attitudes made at an individual level can underestimate the gains that an intervention might make.
|
OBJECTS OF INTEREST IN THE EVALUATION OF COMMUNITY-BASED HEALTH PROMOTION |
|---|
|
TOPSUMMARYINTRODUCTIONVALUES AND RELATED ISSUES...OBJECTS OF INTEREST IN...THE SPIRIT OF THE...THE USE OF STANDARDS...MOVING TOWARD...ACKNOWLEDGEMENTSREFERENCES |
|---|
Before one can set standards for a community-based health programme there is the need to articulate relevant ‘objects of interest’ (i.e. those factors or variables that will be tracked and assessed in a given evaluation). The objects of interest for community-based health promotion programmes or policies can be quite diverse. Below we discuss several issues related to the nature of the community-based health programme as it relates to the question of ‘objects of interest’ for evaluation.
The diversity of potential objects of interest derives first from the fact that health promotion interventions can occur at multiple levels (biomedical, lifestyle or behavioural, and socio-environmental) and in diverse settings within a community. Internationally, community-based health promotion has become a major strategy that has received prominence through major studies [e.g. Stanford Five City Project, PATCH (Planned Approach Towards Community Health), and various Healthy Cities approaches]. Many such projects have faced major challenges and many have not achieved great changes in health (Syme, 1997; Green and Kreuter, 1999; Potvin and Richard, 2001). Programmes have not always been relevant to those being targeted and intervention methods are not always appropriate to those involved. The focus on communities does not always take into account the diverse sub-groups and the social context in which people live and work (Syme, 1997). Nutbeam and his colleagues (Nutbeam et al., 1993) concluded that the information gained in large-scale programmes can disseminate quickly to surrounding jurisdictions and interfere with classic intervention and evaluation designs through contamination.
Secondly, health promotion often employs multiple strategies, including creating healthy public policy and supportive environments, fostering individual or group skills and capacities, strengthening community action and reorienting health services (World Health Organization, 1986). It may try to modify the social context that influences health behaviours as a means to achieve improved quality-of-life and well-being. In this regard, health promotion is wholly consistent with what Hamilton and Bhatti termed ‘population health promotion’ (Hamilton and Bhatti, 1996), and with notions of ‘population health’ and the determinants of health that have come to dominate the health discourse in Canada and elsewhere (Frankish et al., 1999).
Community-based health promotion programmes are often large in scope, have extended time frames and require many resources. We argue that health promotion programmes are most likely to be beneficial when they are flexible and responsive to changing realities. Health promotion programmes and associated evaluations must also accommodate diverse definitions of what the term ‘community’ means. Community has all of the following elements—identity, geography and politics. Finally, they must struggle with issues of representativeness and who can speak for a given community (Wiesenfeld, 1996).
Evaluations of community-based health promotion programmes may be quantitatively and qualitatively distinguished from typical experimental studies to the degree that they embrace a multi-level, multi-strategy vision of individual and environmental change. In this regard, a wide spectrum of evaluation approaches has been used in community health promotion. These incorporate highly structured, methodologically driven evaluations, including randomized control trials (RCTs) through to much less rigidly structured, highly participatory forms of research and evaluation as discussed in the previous section (Nutbeam, 1998). Little consensus has been reached about the most appropriate method of evaluating community-based programmes. Nutbeam encourages an ‘expansionist’ approach to evaluation that would consider the range of strategies employed, the different outcomes from those strategies and provide a wide range of potential indicators of success (Nutbeam, 1998). Consideration of a broad range of measures of success or standards of acceptability fits much more comfortably with modern concepts of health promotion. The development of indicators and instruments that measure these changes is equally important [see (Nutbeam, 1998)].
Based upon the complexity of health promotion programmes delineated in the preceding paragraphs, it is evident that the possible objects of interest in an evaluation are vast. Ideally, community-based health promotion programmes have a balanced emphasis on processes, impacts and outcomes that serve as the objects of interest in their evaluations. They should attempt to integrate ‘process’ evaluation (i.e. intervention activities, staff performance, etc.) with ‘impact’ evaluation (i.e. proximal, intermediate changes in behaviour, lifestyle and the environment) and with ‘outcome’ evaluation (i.e. distal, longer-term changes in policy, health status, etc.). While one need not address every level of evaluation in a single project, it is useful to consider each level from a conceptual and planning perspective. The recent shift toward a ‘population health’ approach in which the objects of interest are distal, non-medical determinants of health further complicates the evaluation process.
The preceding section highlights some of the complexities associated with defining ‘objects of interest’ for health promotion. Such objects of interest to be included in an evaluation need to be clearly delineated at the outset of the process. However, in keeping with our definition of evaluation, the identification of objects of interest is only the first step. Next, programme planners and decision-makers must articulate ‘standards of acceptability’ for each object of interest. That is, they must choose where they will ‘set the bar’ and how they will define the success of each element of a given programme or policy. As Patton states:
‘Objectives are often set a long time before the programme is under way or well before an actual evaluation has been designed. Reviewing objectives and establishing precise standards of ... [acceptability] just before data collection increases the likelihood that judgement criteria will be up to date, realistic, and meaningful’ [(Patton, 1997
If they are unwilling or unable to set standards before data collection in a calm and deliberate manner, there is no reason to believe they can do so afterward (Patton, 1997). One means of facilitating the setting of standards is to create speculative or dummy data for the objects of interest; this makes the process more concrete. The explicit articulation and linking of standards and objects of interest will facilitate a worthwhile evaluation that in turn will improve the programme, and the health of communities.
|
THE SPIRIT OF THE TIMES |
|---|
|
TOPSUMMARYINTRODUCTIONVALUES AND RELATED ISSUES...OBJECTS OF INTEREST IN...THE SPIRIT OF THE...THE USE OF STANDARDS...MOVING TOWARD...ACKNOWLEDGEMENTSREFERENCES |
|---|
In recent times, many health promotion programmes, their evaluations, and standards of acceptability seem to be driven more by a concern for the electoral cycle than by scientific evidence or community relevance. This drive towards ‘accountability’ stems from a public demanding greater responsiveness of health professionals and policy-makers, and concerns regarding allocations of economically pressed health resources by governments, health care providers and organizations (Alexander et al., 1995
; Solberg et al., 1997; Morfitt, 1998; Zakus, 1998). This drive is in tension with a parallel, increased interest in social capital and the role of community-level factors in generating healthy communities (Minkoff, 1997; Rose et al., 1997; Eastis, 1998; Lomas, 1998). It is often in conflict with the idea that programmes may be more effective if they emerge from local consensus and priorities (Health and Welfare Canada, 1990; Tabrizi, 1995; Zakus and Lysack, 1998).
Within the current economic and political climate there is also strong pressure to incorporate what is termed ‘evidence-based practice’—a borrowed medical paradigm that has been applied mainly to clinical decision making. Evidence-based practice is a framework in medicine for asking questions, tracking new types of strong and useful evidence, distinguishing it from weak, irrelevant or useless evidence, and putting it into practice. The concern with this approach is the undue emphasis that is placed on RCTs and meta-analysis (Rada et al., 1999). Not all interventions can be investigated by these methods, nor can they, in the case of health promotion practice, be economically justified. While approaches that consider clustering issues [see (Simpson et al., 1995; Thompson et al., 1997; Hayes and Bennett, 1999)] can come closer to addressing the realities of conducting community-based evaluations, they are not wholly satisfactory.
Most stakeholders support the need for a conceptually sound evidence base for interventions that aim to promote health. However, the current search for evidence using methods and strategies that do not fit with community realities is unlikely to succeed. Health promotion programmes may be at risk of the application of inappropriate methods of assessing evidence, an over-emphasis on health status outcomes and individual behaviour change, and an increased pressure on precious resources (Speller et al., 1997b). These emphases may be to the detriment of important considerations and evidence relating to the building of community capacity and addressing the broader, non-medical determinants of health. It may also result in drawing inappropriate conclusions regarding health promotion practice.
In summary, the evaluation of community-based health promotion programmes differs in substantive ways from controlled experimental studies (see next section). Many of the traditional assumptions of positivist research either do not hold or are very difficult to apply in a community setting. Having recognized these difficulties does not mean that community stakeholders are free from responsibility for evaluation and/or accountability. Rather, there is a need for a balanced approach to evaluation that accommodates both community realities and decision-makers' concerns for evidence and accountability.
The next section presents an integrative approach to setting standards in community-based health promotion. Our purpose is to frame evaluation as a win-win, collaborative and capacity-building exercise.
|
THE USE OF STANDARDS IN EVALUATING COMMUNITY-BASED HEALTH PROMOTION INITIATIVES |
|---|
|
TOPSUMMARYINTRODUCTIONVALUES AND RELATED ISSUES...OBJECTS OF INTEREST IN...THE SPIRIT OF THE...THE USE OF STANDARDS...MOVING TOWARD...ACKNOWLEDGEMENTSREFERENCES |
|---|
We recognize that community-based health promotion (and its evaluation) is a multi-stage process involving the setting of objectives, the execution of strategies, the collection of data, and an assessment or appraisal of the relative success or failure of a given intervention. Glasgow, Vogt and Boles proposed a similar comprehensive evaluation framework, where they argued that multifaceted interventions incorporating policy, environmental and individual components should be evaluated with measurements suited to their settings, goals and purpose (Glasgow et al., 1999
). They proposed the RE-AIM model for evaluating public health interventions that assessed five dimensions: reach, efficacy, adoption, implementation and maintenance. It is the process of stating objectives and associated standards that is of interest here. Three elements are central to our proposed approach. The first is our strong endorsement and adoption of a salutogenic stance and values base. The second is our recommendation for the use of a comprehensive, diverse set of standards that reflect different concerns and forms of evidence. The third is for the use of an inclusive, empowering process of dialogue that engages all relevant stakeholders in the setting of standards for a given initiative.
A ‘standard’, as defined by the Webster's New Collegiate Dictionary (1979) is something established by authority, custom or general consent as a model or example. In the health promotion context, standards of acceptability serve to identify the desired level of outcome and allow all parties to agree on how much change should be achieved in return for a given investment of resources. They serve as targets, which, when met or exceeded, signal success, improvement or growth. Standards can be technically, procedurally, system- or outcome-oriented.
Many fields and disciplines have utilized standards of acceptability (McKenzie and Jurs, 1993; McKenzie and Pinger, 1997; Green and Kreuter, 1999). Similarly, Patton uses the phrase ‘standards of desirability’ to evaluate programmes (Patton, 1997). The use of standards in community-based health promotion is in keeping with the parallel movement toward use of a broad range of community health indicators identified through a collaborative process (Hancock et al., 1998) [see also a special issue of Health Promotion International (1988), 3 (1)]. For health promotion programmes, the standards will be the expected level of improvement in the social, economic, health, environmental, behavioural, educational, organizational or policy conditions stated in the programmes' objectives and represented in the associated objects of interest for evaluation.
The following section identifies eight approaches to setting standards that we argue should be considered in community health promotion. We discuss the relative strengths, weaknesses and applicability of each approach. The different approaches to setting standards are organized according to what Green and Kreuter term the ‘three world views of population needs and planning’ (Green and Kreuter, 1999) (see Figure 1).
|
Arbitrary, experiential and utility standards fall into the upper left circle, in which planning (and evaluation) is primarily driven by the perceived needs, values and expectations of practitioners, lay participants or professional decision-makers. Historical, scientific and normative standards fall into the upper right circle, where planning and evaluation are driven by empirical, objective data. Finally, propriety and feasibility standards fall into the bottom circle, wherein the primary concern is for consideration of available resources, existing policies, legislation and administrative factors. Objective and policy-related standards (which are like scientific standards) are often given greater weight by external decision-makers than those in the upper left circle. The intersection of the three circles represents what has been termed ‘model’ standards (American Public Health Association, 1991
).
Our position is that there is nothing inherently superior about any one of the eight types of standards. Judgment and discretion are unavoidable, and to some degree desirable, in decision making, which operates within a paradigm or environment that shapes the process and the outcome(s). Decision making is a social process and methods are social constructions that are historically determined and situated, and build only on existing knowledge (Potvin et al., 1994; Potvin, 1996). The more important question is: which type of standard fits, with which questions, in what circumstances?
Standards based on perceived needs and priorities
Arbitrary standards
Arbitrary standards are a simply declared or expected level of change, and are most often put forward by individuals or groups in a position of authority. An example of an arbitrary standard is one in which a decision-maker sets the standard for a given initiative without sufficient consultation with important stakeholders and/or consideration of available relevant information.
An advantage of arbitrary standards is the efficient way in which they are created. Some disadvantages include that such standards may be biased in favour of their creator's point of view and the process may be perceived as dictatorial and non-inclusive. For communities, arbitrary standards are often not realistic, and often little ownership or motivation to meet such standards exists. Arbitrary standards are not capacity building, and thus practitioners and the communities they serve are likely to have little commitment to facilitating or participating in such a health-promoting project.
Experiential standards
Experiential standards involve a community's perceived needs and priorities. They recognize the value and utility of local, indigenous knowledge and are community-specific. Their use is in tension with other types of standards (i.e. normative, scientific) that are based on external data or information drawn from other jurisdictions. Some communities may want to emphasize process and/ or unanticipated outcomes. Some decision makers may perceive these issues as a disadvantage in the use of experiential standards. Rodney et al. offer an example of the use of indigenous knowledge as it relates to the evaluation of a community health advocate programme (Rodney et al., 1998). Their use of three interrelated evaluation processes helped to document the need for, and the effectiveness of their programme.
Utility standards
Utility standards are intended to ensure that a community-based health promotion programme will serve the needs of programme recipients, community stakeholders, practitioners and government decision-makers (Joint Committee on Standards for Educational Evaluation, 1994). This approach may include a priori identification of stakeholders and their needs, and the selection of pertinent evaluation questions. Although needs-based or utility standards have the potential advantage of relevance to local circumstances, they may be limited in their representativeness.
Standards based on objective data
Historical standards
Historical standards are based on previous performance and data. Generally, this method applies to outcome objectives that can be easily measured such as attendance at clinics, and birth or mortality rates. They are incremental in nature, and are most useful in situations in which data are routinely accessible.
The use of historical standards has several potential advantages. Practitioners may be more comfortable with these standards because they have been previously involved in devising and/or carrying out these standards. Their skills have been developed, and can build on previous successes. A practitioner's role in the development of historical standards may be of a technical nature, such as collecting and interpreting data.
Historical standards are not necessarily a single point but may represent several points across time, as in trend analysis. The benchmarks may be transparent and repeatedly collected in a consistent manner. For example, Serxner and Chung conducted a trend analysis of social and economic indicators of mammography use in Hawaii (Serxner and Chung, 1992). Systems like the Behavioral Risk Factor Surveillance System offer the necessary longitudinal data. Similarly, Hughes and Cox examined breastfeeding initiation in Tasmania by demographic and socioeconomic factors for the period 1981–1995 (Hughes and Cox, 1999). They noted that trend data is an important component of infant health and nutrition monitoring and surveillance systems. It is also an important basis for identifying breastfeeding promotion needs, prioritizing target groups and strategies, and in evaluating the effectiveness of breastfeeding promotion efforts.
There are several potential limitations or disadvantages to the use of historical standards. They may be skewed and data may not be attributable to a health promotion programme when unique phenomena occur. Such phenomena may be the result of new policies and/or media campaigns within an altered socio-political context. These standards may be flawed if they are based upon inaccurate or biased data. In such incidences, historical standards only serve to replicate an inherent error. By their nature, historical standards are not appropriate for new programmes since there is no pre-existing data.
Normative standards
Normative standards, as with historical standards, are those wherein data such as the state or national average for a given health behaviour is routinely collected. Normative standards are usually based on what other programmes or organizations in similar settings have achieved, with the advantage that these may be used as benchmarks. In this case, the benchmark is a level, and may or may not represent a point in time. To use this method, documentation must be available to practitioners.
In Canada, the British Columbia Ministry of Health (BCMH, 1994) has produced a framework and process for screening for local area benchmarks that involves selective causes of death (eight indicators), lifestyle characteristics (five indicators) and birth factors (four indicators). More recently, many governments have adopted a ‘report card’ approach that reports on the health status of a given population, usually on a year-to-year basis. Associated with such report cards is the parallel proliferation of a host of national, provincial or state databases. Each is intended to provide the requisite data for making normative comparisons and planning programmes or policies. It is important to note, however, that most of these databases are not oriented toward health promotion. Furthermore, indicators of ‘community health’ or community-level indicators are often excluded (Frankish and Bishop, 1999).
Normative standards may provide a clear point of reference for health promotion planners and are most likely to be based on ‘objective’ (quantitative) data. These provide a measure of efficiency because practitioners are ‘not re-inventing the wheel’. If they have been used successfully elsewhere as a standard for evaluation, they may allow for comparative interventions across jurisdictions. Qualities of credibility, efficiency and feasibility are often associated with normative standards and may enhance the probability that health promotion planners will endorse this type of standard.
There are several potential limitations to the use of normative standards. For communities or states, normative standards set in relation to other jurisdictions may be unrealistic and/or unachievable, and may not represent a priority focus for a specific jurisdiction. A further practical difficulty is that of finding an appropriate comparison community or jurisdiction. In some cases, the appropriateness of using one community's achievements for another community's standard can be questioned. Some communities (i.e. Aboriginal or low socioeconomic groups) have become frustrated with evaluation reports continually positioning them at the bottom. Others question the feasibility of generating community-specific data in order to demonstrate a normative comparison.
Finally, there are ethical concerns related to the question ‘what makes a fair comparison across communities?’. Fair is a relative term, and is dependent upon the resources at one's disposal. In this regard, it is important to make a distinction between responsibility and reliance. Communities and health promotion practitioners can only be expected to meet standards that are consistent with available resources and capacities. Ideally, an evaluation process can assist communities to be more self-reliant and responsible with the resources they do have or to acquire additional resources.
Scientific standards
Scientific standards may be empirically and/or theoretically based, and are developed from outcomes achieved in controlled studies and generally based on systematic reviews of available literature. Such standards place emphasis on RCTs and meta-analysis (Rada et al., 1999). Recent examples include the movement toward ‘best practices’ (Sherman, 1999), the development of ‘preventive practice guidelines’ (US Preventive Services Task Force, 1996) and systematic research syntheses of the type associated with the Cochrane collaborations and databases.
The major advantage of scientific standards is that they are viewed as objective, empirical and unbiased. They align with a dominant view of ‘evidence’, which suggests that such standards are more credible and trustworthy than data or evidence generated by other means (i.e. qualitative methods). From a positivist perspective, this ‘gold’ standard is only achievable through empirical science of the type associated with RCTs.
Several disadvantages exist in trying to apply ‘scientific’ standards to community-based health promotion programmes or policies. Such settings make it impossible to randomly assign individuals or groups to a particular community, and it is sometimes difficult to identify appropriate comparison or control communities. When the unit of analysis is an entire community (rather than an individual) it is difficult to manifest the level of ‘control’ desired in a typical scientific study. In fact, the complexity of factors associated with community life is a key to the dynamics of community-based health promotion. Attempting to isolate single variables is contrary to notions of holism, reciprocal interactions and interdependence associated with communities.
The use of scientific standards in community-based health promotion may be perceived as arbitrary, and their ‘goodness of fit’ to the circumstances or needs and expectations of a given community is questionable. There are also ethical questions inherent in the notion of ‘control’ communities. Holding some components of a community's capacities constant is contradictory to the empowering, skill-developing process of community-based health promotion.
Scientific standards, when imposed by external decision-makers (e.g. government or funders) are a source of tension for most practitioners and many health promotion theorists. Randomized control trials are time-consuming, expensive, and require a skill level many practitioners do not possess. Community practitioners may not have access to relevant data, such as the latest published evaluations, which are most often contained in academic journals. From a policy perspective, government decision-makers may not be able or want to wait for ‘scientific’ data to be generated.
Standards based on available resources and existing policies
When it comes to setting standards for community-based health promotion programmes, planners, practitioners and government decision-makers must consider different options with respect to data, evidence and benchmarks. They may also consider practical issues such as existing policies, regulations and legislation, logistical factors and the availability of resources.
Propriety standards
Propriety standards are intended to ensure that community-based health promotion programmes are conducted legally, ethically and with regard to the welfare of community participants (Joint Committee on Standards for Educational Evaluation, 1994). Issues such as formal agreements, fiscal responsibility and conflict of interest are relevant in consideration of propriety standards [see (Roman and Blum, 1987; Jacob, 1994; Starzomski, 1995; Jenkins and Emmett, 1997)]. Brown provides an example of propriety standards in relation to environmental health issues and the US Congress debate over a ‘polluter-pay’ approach to dealing with violations of existing legislation (Brown, 1997).
Feasibility standards
Feasibility standards are intended to ensure that the programme will be realistic, prudent and frugal (Joint Committee on Standards for Educational Evaluation, 1994). Feasibility involves considerations of cost effectiveness, political viability and practical procedures. One advantage of including feasibility standards is that they may serve as a ‘reality check’ with respect to available resources. They may also act as a catalyst for securing additional resources. One potential disadvantage is that a ‘bottom-line’ mentality may undermine innovation and creativity. Richardson questioned the common belief that economic evaluation is hostile to health promotion and that the requirement for health programmes to be cost effective will result in a biased allocation of funds in favour of programmes that can demonstrate short-term benefits as defined by inadequate outcome measures (Richardson, 1998). He notes the potential for economic evaluation to be counter-productive if applied to ‘immature’ projects, and the practical problems inherent in the measurement of outcomes in health promotion programmes. He proposes a four-fold classification based on a distinction between disease cure, individual health promotion, community welfare and systemic change designed to promote either individual health or social well-being.
Van der Weijden and her colleagues analysed the feasibility of using national cholesterol guidelines in general medical practices (Van der Weijden, 1999). Their programme was developed after barriers to working according to the guideline had been investigated. The quality of targeting of cholesterol testing did not improve following the intervention. This research demonstrated that neither simple dissemination nor an intensive programme had a measurable impact on performance of work according to the cholesterol guideline. Stephenson et al. assessed the feasibility of conducting a large RCT of peer-led intervention in schools to reduce the risk of HIV/ STDs and promote sexual health (Stephenson et al., 1998). Questionnaire completion rates of 90% indicated considerable enthusiasm for peer-led education among educators and pupils. Evaluation of the behavioural intervention was shown to be acceptable to schools, pupils and parents, and feasible in practice.
A composite approach to setting standards: model standards
The section above highlights a variety of approaches to setting standards that are relevant to the practice and evaluation of community-based health promotion. The presentation of the eight types of standards recognizes that the various approaches are not mutually exclusive, nor are they independent. The diversity of approaches does beg the question of how different approaches might be combined.
One method of combining a variety of standards is the so-called ‘model standards’ approach. This approach is an amalgam and incorporates elements of each of the other types of standards. The term ‘model’ standard is associated with a specific approach developed in the United States in response to Healthy People 2000 and Healthy Communities 2000 through the cooperation of communities, local health agencies and the private sector (APHA, 1991). Similar to its original usage, our use of the term ‘model’ is not intended to connote that, in and of itself, this approach represents the ‘optimal’ or best approach to setting standards in community-based health promotion.
With the US approach, model standards were developed to plan programmes and to allocate resources. As a companion to the Healthy People 2000 report, these standards offer community implementation strategies for putting objectives into practice by establishing achievable community health targets. This method adapts national targets for local relevance and suggests an array of activity-based objectives. In the American approach, a lead agency, such as the local health department, drives the process of articulating ‘model’ standards by organizing the effort and providing the needed technical expertise in relevant public health practice. The use of a lead agency approach may, however, raise issues of control, questions about roles and responsibilities, and has the potential for disempowerment of the community members.
Model standards have also been used elsewhere (Speller et al., 1997a). A project to develop a framework for quality assurance in health promotion practice in England has recently been developed. Six key functions of health promotion (strategic planning, programme management, monitoring and evaluation, education and training, resources and information, and advice and consultancy) were identified. Model standards and criteria were drawn up for each function, together with guidance on implementation processes.
Model standards may be expressed as programme processes, risk factors or objectives related to a specific health outcome. These standards need to be flexible to accommodate differences in the mix of contexts and services available. Stakeholders can therefore participate in determining their own public health priorities that are compatible with national objectives and targets.
These standards represent a form of compromise or consensus standards. They are generally established from a consensus of informed opinions by professionals and experienced others, and may also have the endorsement of professional organizations. A disadvantage of these standards may be the time taken to generate them.
Compromise standards may be political in nature and depend on the quality of the people involved. Individuals or specific stakeholder groups may come to the table with diverse and sometimes competing/hidden agendas. While ‘model’ standards suggest an optimal mix of standard setting approaches, for some these standards may be settling for the lowest common denominator.
|
MOVING TOWARD ‘OPTIMAL’ STANDARDS FOR COMMUNITY-BASED HEALTH PROMOTION |
|---|
|
TOPSUMMARYINTRODUCTIONVALUES AND RELATED ISSUES...OBJECTS OF INTEREST IN...THE SPIRIT OF THE...THE USE OF STANDARDS...MOVING TOWARD...ACKNOWLEDGEMENTSREFERENCES |
|---|
This paper addresses issues related to evaluation and the use of standards in community-based health promotion. These issues include the definition and measurement of relevant outcomes and the use of participatory, empowering evaluation methodologies that assess both the outcomes achieved and the processes by which they are accomplished.
We recognize that considerable progress has been made in understanding the complexity of undertaking evaluations in community settings. We acknowledge the corresponding need for tools, measures and evaluation designs that accommodate this complexity.
Finally, we recognize two realities. First, that good science poorly applied will not advance the quality and utility of community-based evaluations. There is little benefit to be gained from forcing RCT-type designs to be used in circumstances where they do not fit. Both the process and outcomes of community-based evaluations must be relevant to community stakeholders, policy-makers and/or funders. Secondly, the ‘balloons and t-shirts’ approach to community-based health promotion programmes, in which there is little or no attention paid to evaluation, is equally inappropriate. Policy-makers, funders and taxpayers have a right to demand accountability and some measure of the success of health promotion initiatives.
Our taxonomy of standards, grounded in a salutogenic values stance, is offered as a potential means of bridging these ‘two solitudes’. The hope is to create a win-win situation in which policy-makers and funders are more supportive of evaluation designs (i.e. processes and outcome measures) that fit with community realities, and community stakeholders are more capable and consistent in evaluating community-based health promotion programmes and policies.
We advocate a shift away from a view of evaluation that is dominated by a pathogenic, risk factor and outcomes-oriented perspective toward a more balanced menu of possible targets for change and accompanying standards for defining success. This suggestion is not at odds with standards that are systematic and supportive of accountability. We conclude by recommending that each of our eight types of standards [arbitrary, experiential (community), utility, historical, scientific, normative, propriety and feasibility] be considered in planning the evaluation of community-based health promotion programmes or policies. Explicit consideration of this diverse set of standards may be used to engage all stakeholders in inclusive, empowering dialogue. It demands that stakeholders’ respective concerns, views of evidence and definitions of success be examined. In the end, ‘optimal’ standards for community-based health promotion will be those that engage diverse stakeholders in a process of collaborative dialogue and decision making. They will maximize the fit of the evaluation process and targets with community capacities, perspectives and resources. Finally, optimal standards will help to yield new knowledge that will contribute to health, well-being and quality of life of individuals, families and communities.
Our hope is that collaborative evaluations will take into account the varying nature of communities while building social capital, community capacity, economic viability and well-being. Well formulated evaluations can assist funders, policy-makers, practitioners and communities in linking the success of specific programmes or policies to broader contextual economic, environmental or social issues.
|
ACKNOWLEDGEMENTS |
|---|
|
TOPSUMMARYINTRODUCTIONVALUES AND RELATED ISSUES...OBJECTS OF INTEREST IN...THE SPIRIT OF THE...THE USE OF STANDARDS...MOVING TOWARD...ACKNOWLEDGEMENTSREFERENCES |
|---|
This work was completed while the first author was a Visiting Student at the Institute of Health Promotion Research (IHPR), University of British Columbia and a Doctoral student in Health Science at Deakin University, Australia. The authors wish to acknowledge the support of Territory Health Services—Long Service Leave (Darwin, Australia) and Health Canada. They also wish to recognize the support of their colleagues in the Institute of Health Promotion Research. From Australia, Penny Hawe, Lawry St Leger and Sandy Gifford provided valuable comments on an earlier version of this manuscript.
|
REFERENCES |
|---|
|
TOPSUMMARYINTRODUCTIONVALUES AND RELATED ISSUES...OBJECTS OF INTEREST IN...THE SPIRIT OF THE...THE USE OF STANDARDS...MOVING TOWARD...ACKNOWLEDGEMENTSREFERENCES |
|---|
Alexander, J., Zuckerman, H. and Pointer, D. (1995) The challenges of governing integrated health care systems. Health Care Management Review, 20, 69–92.[ISI][Medline]
American Public Health Association (APHA) (1991) Healthy Communities 2000: Model Standards. American Public Health Association, Washington, DC.
Antonovsky, A. (1979) Health, Stress and Coping. Jossey-Bass, San Francisco.
Antonovsky, A. (1996) The salutogenic model as a theory to guide health promotion. Health Promotion International, 11, 11–18.
British Columbia Ministry of Health and Ministry Responsible for Seniors (1994) Processes, Benchmarks and Responsibilities for Developing Community Health Councils and Regional Health Boards: Meeting the Challenge, Action for a Healthy Society. BC Ministry of Health and Ministry Responsible for Seniors, Victoria, BC.
Brown, K. S. (1997) Off the hook: what Olin might mean. Environmental Health Perspectives, 105, 44–47.[ISI][Medline]
Camiletti, Y. A. (1996) A simplified guide to practicing community-based/community development initiatives. Canadian Journal of Public Health, 87, 244–247.[ISI][Medline]
Cervero, R. M. and Wilson, A. L. (1994) The politics of responsibility: A theory of program planning practice for adult education. Adult Education Quarterly, 45, 249–268.[Abstract]
Cowley, S. and Billings, J. R. (1999) Resources revisited: salutogenesis from a lay perspective. Journal of Advanced Nursing, 29, 994–1004.[ISI][Medline]
Cox, E. (1997) Building social capital. Health Promotion Matters, 4, 1–4.
Eastis, C. (1998) Organizational diversity and the production of social capital. American Behavioral Scientist, 42, 66–77.[Abstract]
Eisen, A. (1994) Survey of neighbourhood-based comprehensive community empowerment initiatives. Health Education Quarterly, 21, 235–252.[ISI][Medline]
Fetterman, D. M., Kaftarian, S. J. and Wandersman, A. (eds) (1996) Empowerment Evaluation: Knowledge and Tools for Self-Assessment and Accountability. Sage Publications, Thousand Oaks, CA.
Frankish, C. J. and Bishop, A. (1999) Background Paper and Plan for Inclusion of Community Health Indicators in the Canadian Community Health Survey. Prepared for the Canadian Consortium of Health Promotion Research Centres and the Advisory Committee on the Canadian Community Health Survey, Ottawa, ON.
Frankish, C. J., Veenstra, G. and Moulton, G. (1999) Population Health in Canada: a Working Paper. Prepared for the National Conference on Shared Responsibility for Health and Social Impact Assessments: Advancing the Population Health Agenda. Institute of Health Promotion Research, University of British Columbia, Vancouver, BC.
Glasgow, R. E., Vogt, T. M. and Boles, S. M. (1999) Evaluating the public health impact of health promotion interventions: the RE-AIM framework. American Journal of Public Health, 89, 1322–1327.
Green, L. W. and Kreuter, M. W. (1999) Health Promotion Planning: an Educational and Ecological Approach, 3rd edn. Mayfield Publishing Company, Mountain View, CA.
Green, L. W., George, M. A., Daniel, M., Frankish, C. J., Herbert, C. J., Bowie, W. R. and O'Neill, M. (1995) Study of Participatory Research in Health Promotion: Review and Recommendations for the Development of Participatory Research in Health Promotion in Canada. Institute of Health Promotion Research, University of British Columbia and the BC Consortium for Health Promotion Research for The Royal Society of Canada, Vancouver, BC.
Guba, E. G. and Lincoln, Y. S. (1989) Fourth generation evaluation. Sage, Newbury Park, CA.
Hamilton, N. and Bhatti, T. (1996) Population Health Promotion: an Integrated Model of Population Health and Health Promotion. Health Canada, Ottawa, ON.
Hancock, T., Labonte, R. and Edwards, R. (1998) Indicators that Count!—Measuring Population Health at the Community Level. Health Canada, Ottawa, ON.
Hawe, P. (1994) Capturing the meaning of community in community intervention evaluation: some contributions from community psychology. Health Promotion International, 9, 199–210.
Hawe, P., King, L., Noort, M., Gifford, S. M. and Lloyd, B. (1998) Working invisibly: health workers talk about capacity building in health promotion. Health Promotion International, 13, 285–294.
Hayes, R. J. and Bennett, S. (1999) Simple sample size calculation for cluster-randomized trials. International Journal of Epidemiology, 28, 319–326.
Health and Welfare Canada (1990) Prevention through Partnership: Collaborating for Change. National Strategy for Enhancing Preventive Practices of Health Professionals. Report of a National Workshop, October 28–31, 1990, Ottawa. Health and Welfare Canada, Ottawa, ON.
Health Canada Population Health Directorate (1996) Guide to Project Evaluation: a Participatory Approach. Available at: http://www.hc-sc.gc.ca/hppb/phdd/guide/introduction.htm. Accessed on October 21, 1999.
Hughes, R. and Cox, S. (1999) An analysis of breastfeeding initiation in Tasmania by demographic and socioeconomic factors for the period 1981–1995. Breastfeeding Review, 7, 19–23.
Israel, B., Checkoway, B., Schulz, A. and Zimmerman, M. (1994) Health education and community empowerment: conceptualising and measuring perceptions of individual, organisational, and community control. Health Education Quarterly, 21, 149–170.[ISI][Medline]
Jacob, F. (1994) Ethics in health promotion: freedom or determinism? British Journal of Nursing, 3, 299–302.[Medline]
Jenkins, D. and Emmett, S. (1997) The ethical dilemma of health education. Professional Nurse, 12, 426–428.
Joint Committee on Standards for Educational Evaluation (1994) The Program Evaluation Standards: How to Assess Evaluations of Educational Programs. Sage Publications, Thousand Oaks, CA.
Kawachi, I., Kennedy, B., Lochner, K. and Prothrow-Stith, D. (1997) Social Capital, Income inequality and mortality. American Journal of Public Health, 87, 1491–1498.
Labonte, R. (1993) Health Promotion and Empowerment: Practice Frameworks. Participaction Series. Centre for Health Promotion, University of Toronto, Toronto, ON.
Labonte, R. (1996) Health promotion and population health: what do they have to say to each other? Canadian Journal of Public Health, 86, 165–167.
Labonte, R. and Robertson, A. (1996) Delivering the goods, showing our stuff: the case for a constructivist paradigm for health promotion research and practice. Health Education Quarterly, 23, 431–447.[ISI][Medline]
Lomas, J. (1998) Social capital and health: implications for public health and epidemiology. Social Science and Medicine, 47, 1181–1188.
McKenzie, J. F. and Jurs, J. L. (1993) Planning, Implementing and Evaluating Health Promotion Programs: a Primer. Macmillan Publishing Company, New York.
McKenzie, J. F. and Pinger, R. R. (1997) An Introduction to Community Health. Jones and Bartlett Publishers, Sudbury, MA.
Minkoff, D. (1997) Producing social capital—national social movements and civil society. American Behavioral Scientist, 40, 606–619.[Abstract]
Morfitt, G. (1998) Report of the Auditor General on Regionalization, Accountability and Governance. Auditor General's Office, Victoria, BC.
Nutbeam, D. (1998) Evaluating health promotion—progress problems and solutions. Health Promotion International, 13, 27–44.
Nutbeam, D., Smith, C., Murphy, S. and Catford, J. (1993) Maintaining evaluation designs in long term community based health promotion programmes: Heartbeat Wales case study. Journal of Epidemiology and Community Health, 47, 127–133.[Abstract]
Patton, M. Q. (1997) Utilization-Focused Evaluation: The New Century Text, 3rd edn. Sage, Thousand Oaks, CA.
Potvin, L. (1996) Methodological challenges in evaluation of dissemination programs. Canadian Journal of Public Health, 87, S79–S83.
Potvin, L. and Richard, L. (2001) The evaluation of community health promotion programs. In Rootman, I., Goodstadt, M., Hyndman, B., McQueen, D. V., Potvin, L., Springett, J. and Ziglio, E. (eds) Evaluation in Health Promotion: Principles and Perspectives. World Health Organization, Copenhagen.
Potvin, L., Paradis, G. and Lessard, R. (1994) Le paradoxe de l’évaluation des programmes communautaires multiples de promotion de la santé. Ruptures, 1, 45–57.
Putnam, R. (1993) Making Democracy Work: Civic Traditions in Modern Italy. Princeton University Press, Princeton, NJ.
Rada, J., Ratima, M. and Howden-Chapman, P. H. (1999) Evidence based purchasing of health promotion: methodology for reviewing evidence. Health Promotion International, 14, 177–187.
Raeburn, J. and Rootman, I. (1998) People-Centred Health Promotion. John Wiley & Sons, Toronto, ON.
Richardson, J. (1998) Economic evaluation of health promotion: friend or foe? Australian and New Zealand Journal of Public Health, 22, 247–253.[ISI][Medline]
Rodney, M., Clasen, C., Goldman, G., Markert, R. and Deane, D. (1998) Three evaluation methods of a community health adv