Health Promotion International, Vol. 16, No. 3, 261-268,
September 2001
© Oxford University Press 2001
Strengthening the evidence base for health promotion
National Center for Chronic Disease Prevention and Health Promotion, Centers for Disease Control and Prevention, Atlanta, Georgia, USA
Address for correspondence: David V. McQueen National Center for Chronic Disease Prevention and Health Promotion (NCCDPHP) Centers for Disease Control and Prevention (CDC) MS K40 4770 Buford Hwy NE Atlanta Georgia 30341 USA E-mail: dvmcqueen{at}cdc.gov
SUMMARY
This paper describes the evidence debate from the many players currently attempting to define best practices in health promotion. Expert opinions on the purpose of collecting evidence range from those who view evidence as a western notion of little use in the developing world to those who choose to focus on opportunities to demonstrate the effectiveness of health promotion. There is also much disagreement on what constitutes evidence. Some view evidence as strict outcomes of randomized clinical trials (RCT) and others place greater value on other unpublished sources, not traditionally viewed as valuable information. A challenge for health promotion in the new century is to foster and develop high quality, widely recognized and acceptable standards for evidence-based evaluation.
Key words: evidence; health promotion; public health
A piece of evidence is a fact or datum which is used, or could be used, in making a decision or judgement or in solving a problem. The evidence, when used with the canons of good reasoning and principles of valuation, answers the question why, when asked of a judgement, decision or action (Butcher, 1998
).
INTRODUCTION
Few topics in the field of health promotion have engendered as much heated debate as that of evidence. The spectrum of opinion is broad and diverse. Many believe that health promotion will fall or rise based on its ability to demonstrate in a scientific way that it is an effective field. In contrast, there are those who believe that ‘evidence’, the very word, is inappropriate to the field of health promotion. Between those two extremes are those who question the appropriateness of some forms of evidence and look at the wider role of evidence in terms of knowledge development in the field. These various opinions were largely reiterated by discussions at the Fifth Global Conference on Health Promotion held in Mexico City in the summer of 2000. Despite these contrasting opinions, many practitioners and advocates of health promotion feel a need either to justify their actions or demonstrate to others that health promotion is a field with tangible benefits to offer the public. The role of evaluation and evidence in this justification remains a critical concern.
This paper illustrates some of the broad and diverse dimensions of the debate on the role of evidence in health promotion. Nonetheless, it merely alludes to the rich depth of this debate and the reader is encouraged to pursue this depth in the many documents and Internet forums that now exist. Discussions that took place at the Mexico conference are summarized along with suggestions for the development of a more broad-based view of the evidence debate. It is the hope of many that the pursuit of evidence will further enhance the argument for the effectiveness and importance of the field of health promotion in public health.
Evidence: its nature and relevance to health promotion
The importance of evidence as a topic for health promotion practice should be seen in a larger context of discussions on evidence-based medicine taking place in much of the world. While the debate stems from a fundamental concern with medical practice, it cannot be dismissed as pertinent only to medicine. Health promotion is also challenged by the debate (Adrian et al., 1994; Allison and Rootman, 1996; Macdonald et al., 1996; Sackett et al., 1996; Nutbeam, 1998). Today, health promotion practitioners and researchers are urged to base their work on evidence. In May 1998, the 51st World Health Assembly urged all Member States to ‘adopt an evidence-based approach to health promotion policy and practice, using the full range of quantitative and qualitative methodologies’ (WHA, 1998).
This paper notes at the outset the cultural/ geographical bias of evidence as an idea. Notions such as ‘evidence’, ‘effectiveness’ and ‘investment’ are rightly viewed as western-derived, European–American, and in many ways European language concepts. Thus, most of those who have written and write about evidence have inherent western biases. These biases developed largely out of philosophical conjectures of the past two centuries, notably from debates around logical positivism arising from the influence of Russell and Wittgenstein (Suppe, 1977; Bhaskar, 1997). Logical positivism operates on the tenet that meaning is only verifiable through rigorous observation and experiment. In this context, the word evidence has a very strict analytical meaning. Similarly, the randomized controlled clinical trial (RCT) and the quasi-experimental approach are largely creations of western literature and reflect a reification of the positivist notion. Many social sciences, particularly anthropology and sociology, have alternative, but nonetheless western-derived approaches to assessing evidence and the effectiveness of interventions. To the extent that health promotion uses a mixture of approaches, it is therefore difficult to define clearly the meaning of an evidence approach in health promotion.
There is a strong cultural/geographical bias with the notion of evidence in published literature. Much of the relevant material that could broaden the discussion on evidence is unpublished. The usual western-based computerized sources of research literature citations yield little of this potential breadth. If there are alternative approaches to the issues of evidence from developing countries, they are not readily accessable on the global Internet. Yet, the Internet is a hope for the future once global access to it becomes more readily available. At the Mexico meeting there was a widely held belief among many attending that there was a strong evidence base in developing countries, particularly concerning the evaluation of community programmes. Therefore, a major concern of many was how to disseminate this story effectively, and recommendations were made as to how to make this feasible.
There is another consideration: the evidence debate is not relevant outside the West. Or perhaps it is a mere luxury of excess resources to evaluate health promotion programmes when there is so great an immediate need to implement programmes to tackle pressing health problems.
Several key questions remain:
- Should health promotion programmes in the developing world simply proceed with the assumption that they will use approaches shown to have met evidence criteria drawn up in the West?
- Should there be caution in accepting an evidence criterion for health promotion?
- Can developing countries, in their search for best practice, offer guidance on how best to evaluate programmes with minimal resources?
- Would other approaches be useful and/or transportable to those many western countries with great inequities in population health?
Addressing these questions is not easy, but they need to be recognized as legitimate concerns. Such questions arose in the Mexico conference, but were not resolved.
Whose evidence? Whose rules?
Health promotion prides itself on being eclectic and multi-disciplinary. It is the great strength of the field that it cuts across sectors and disciplines (WHO, 1984). It regards pragmatism as a great value. Multiple approaches that can be used to improve health, reorientate the health care system and empower people are welcomed. Many of the principal activities of health promotion pertain to advocacy, partnerships and coalition building; areas considered art rather than science. Therefore, it may be argued that health promotion is a field of action, highly applied, and having few characteristics of a discipline. Furthermore, it is relatively new as a concerted field of action, still defining its terms, e.g. the word ‘evidence’ does not appear in the recent WHO Health Promotion Glossary (WHO, 1998). Nevertheless, it may also be argued that the field is established in some dimensions. There are institutes, schools, departments, buildings, professorships and programmes named with the term ‘health promotion’.
What does this multi- yet non-disciplinary nature of health promotion have to do with the evidence discussion? Simply put, when one looks for evidence and ‘rules of evidence’ to demonstrate the success or effectiveness of health promotion, one has to retreat into other disciplines for notions of evidence. Thus, in searching for evidence one does not really evaluate the evidence for health promotion, but the effectiveness of a disciplinary subcomponent of the health promotion effort. This results in a highly reductionistic perspective of evaluation. The practice of health promotion, one can argue, presents the dilemma that it is only fairly evaluated on its own terms, i.e. holistically.
Recognizing this dilemma, this paper shifts to the successful evaluation of health promotion practice on other terms, that is the terms of individual disciplines rather than on terms yet to be well articulated by the field of health promotion practice. This is not necessarily a bad thing. Many people who work with health promotion, especially those who need to be convinced of its importance and effectiveness, are discipline based. Thus, there is a good rationale for respecting the ‘rules of evidence’ put forward by scientists working in public health when making the case for the effectiveness of health promotion. At the same time, there is the need to respect and assist in the development of health promotion's own efforts to define the field of evaluation in health promotion.
Defining evidence: a middle ground
The word ‘evidence’ is commonly used to denote something that makes another thing evident (e.g. the fact that a lake is frozen solid is an indication or sign that temperatures have been very low). Words such as apparent, manifest, obvious, palpable, clear or plain may also be used when describing things or events that are evident and all share the characteristics of certainty. In many ways this is a very strict definition. No-one can fail to perceive that which is evident. In brief, the everyday use of the word evidence carries very high expectations; can there be any doubt if one has evidence?
In legal terms, evidence has other meanings. In a trial, for example, evidence is introduced as a way to prove that something has occurred with certainty, even though sometimes this may be just beyond a reasonable doubt. Juries and judges weigh the strength of evidence before rendering a finding of guilt or innocence. Thus, we encounter expressions such as ‘the overwhelming evidence leads us to conclude, beyond a shadow of doubt, that the accused is guilty’. Such language seems foreign to the field of health promotion, but this is exactly the framework espoused by Tones in a recent editorial.
Accordingly, I would argue that we should assemble evidence of success using a kind of ‘judicial principle’—by which I mean providing evidence that would lead to a jury committing themselves to take action even when 100% proof is not available (Tones, 1997
What is interesting about evidence presented in a western legal setting is that it is often a mixture of stories, witness accounts, police testimony, expert opinions and forensic science; in short, it frequently comes from multiple sources and from persons of widely ranging expertise. In this sense, evidence requires interpretation of accounts that vary in their ontological origins.
Evidence can also be understood as derived from systematic observation, sometimes over a long period of time. Often such evidence will be described as empirical or as evidence from observation; sometimes it will be described as evidence from experiment. Sometimes the observations have an underlying theoretical perspective, such as in Darwin's observations leading to a theory of evolution (Weiner, 1995). Observation as evidence is often tied to the notion of data as evidence and this usage is quite common in public health.
Definitions of evidence for health promotion from other sources
Currently, the terms ‘evidence’ and ‘evidence-based’ are regularly invoked in health promotion discussions. Why should this be so? A partial answer may be that evidence has become common currency in population health discussions as well as in discussions of the evaluation of everyday medical practice, particularly preventive practice. Thus, in the West, a sociopolitical climate has permitted a discussion to flower, with emphasis on health care as one major element. Activity has been widespread
In the United States, the Centers for Disease Control and Prevention (CDC, 2000) has taken the lead in assisting an independent Task Force to produce a Guide to Community Preventive Services: Systematic Reviews and Evidence-Based Recommendations, (called the Guide). The Guide intends to define, categorize, summarize and rate the quality of evidence on the effectiveness of population-based interventions to impact on specific outcomes. The Guide will summarize what is known about the effectiveness and cost-effectiveness of population-based interventions for prevention and control, provide recommendations on these interventions and methods for their delivery based on the evidence, and identify a prevention research agenda. This effort is an example of an approach that takes a strong biomedical/epidemiological definition of evidence.
Recommendations in the Guide regarding the interventions are based on the strength of evidence of effectiveness, harms and generalizability. To categorize the strength of a body of evidence on the effectiveness of a specific intervention to impact on a given outcome, the Task Force considers the following factors: (i) suitability of evaluation design to attribute, with confidence, a change in an outcome caused by the given intervention; (ii) quality of study execution; (iii) numbers of studies; (iv) consistency of findings; (v) size of observed effects; and, in rare circumstances, (vi) expert opinion (SAJPM, 2000). The Task Force has noted that population-based prevention strategies are frequently multiple-component and complex, and that randomized controlled trials may not always be feasible or desirable to evaluate the effectiveness of community interventions. What is noteworthy about this undertaking is the time and effort that has gone into establishing a way to categorize and define evidence. In the case of this Task Force, much effort has gone into defining evidence in terms of how interventions are designed.
The scope and size of the task taken on by the Guide is huge. There are 15 members of the Task Force, chosen because of their broad knowledge of public health, preventive medicine and health promotion. They are an independent body with representatives from local health departments, health care organizations, NGOs and universities. In addition, four consultants are attached to the Task Force. The Task Force is supported at CDC by a staff of some 30 senior researchers, research assistants and administrative workers. In addition, there are 14 federal agency liaison members, 17 organization liaison members, and 14 liaison representatives of the CDC offices, institutes and centers. The task force has been in operation since 1997 and continues to pursue the development of the Guide.
The key to the discussion of evidence with respect to the Guide is found in a recent paper summarizing the methods used by the Task Force and the Guide staff:
In the Guide, the term evidence includes: (1) information that is appropriate for answering ques-tions about an intervention's effectiveness; (2) the applicability of effectiveness data (i.e. the extent to which available effectiveness data is thought to apply to additional populations and settings); (3) the intervention's positive or negative side effects (referred to as other effects; includes harms and positive or negative nonhealth outcomes); (4) economic impact; and (5) barriers to implementation of interventions [(SAJPM, 2000), p. 36].
The Guide's thorough approach involves a careful delineation of the process of defining evidence, developing a methodology to assess evidence and then producing recommendations, but it is a tedious and involved process. For example, in the reviews of evidence regarding interventions to improve vaccination coverage in children, adolescents and adults, >250 studies were reviewed, evaluated and examined by a team of some 16 abstractors using a lengthy, detailed and rigorous evaluation form. Even so, this search for evidence was limited to published literature accessible to data retrieval systems such as Medline, Embase, Psychlit, CAB Health and Sociological Abstracts. Furthermore, it only considered publications that were written in English, published between 1979 and 1998, and conducted in industrialized countries, and studies that met the evidence criteria laid out by the Guide team (SAJPM, 2000). This is not to criticize the effort, but rather to emphasize that even a large-scale project has necessary limitations.
The IUHPE report to the European Commission
With the guidance of the International Union for Health Promotion and Education (IUHPE), an advisory group consisting of 13 senior people in the health promotion field, 15 authors and a ‘witness group’ of some 25 ‘political experts’ produced a report for the European Commission (EC) on the evidence of health promotion effectiveness (IUHPE, 1999). The large value of this report, which should be required reading for those interested in the field of health promotion, is that it identifies a considerable body of evidence pointing to the value of health promotion and attesting to its effectiveness. The report was also clear to map out those areas where more research was needed and areas that were open to debate about effectiveness, as well as those areas where health promotion actions have made a difference. Some areas of health promotion activity stand out as of unquestionably powerful value. For example, evidence shows there is a powerful inverse relationship between price and use of tobacco. Therefore, health promoting efforts that lead to price increases of tobacco will lead to less use of tobacco. This finding mirrors that from the CDC group working on tobacco for the community Guide. Thus, there is an accumulating international evidence base for global efforts to reduce tobacco consumption through pricing.
Other areas of health promotion activity require careful thought and further analysis to reveal efficacy. For example, transportation policies impact on health in many ways. However, demonstrating the efficacy of such policies is difficult. In this case, complexity begins to play a major role. While many may believe that there is a highly probable relationship between transportation policy and the general health of a population, the evidence mechanisms to prove any scientific basis for this belief still needs refinement. As a first step, a distinctive logic model or logic framework needs to be developed for each area of health promotion intervention. This logic model maps out the links between social, environmental and biological determinants, and related interventions. These models then serve as a guide for assessing where the evidence challenges are. The challenge for health promotion is to discover how apparently true relationships, such as that between transport policy and health, can be revealed.
Despite all the difficulties with the notion of evidence, the writers of the EC report concluded that:
... evidence clearly indicates that: 1) comprehensive approaches using all five Ottawa strategies are the most effective; 2) certain ‘settings’ such as schools, workplaces, cities and local communities offer practical opportunities for effective health promotion; 3) people, including those most affected by health issues, need to be at the heart of health promotion action programmes and decision making processes to ensure real effectiveness; 4) real access to information and education, in appropriate language and styles, is vital; and 5) health promotion is a key ‘investment’—an essential element of social and economic development (IUHPE, 1999).
What policy makers define as evidence in health promotion
This is a topic that has been taken up extensively. A multi-disciplinary working group on health promotion evaluation consisting of representatives from Health Canada, CDC, WHO (EURO) and HEA discussed, in multiple meetings during 1995–1998, how policy makers view evidence. In the end, a small pamphlet for policy makers was prepared and widely distributed (EWG, 1998). The full deliberations of this evaluation working group are now published (Rootman et al., 2000). A section of the final chapter concludes:
The state of evidence regarding the effectiveness of health promotion is somewhat discouraging. Relatively few evaluations of health promotion have been undertaken, at least as judged by the published literature; in particular, systematic reviews of health promotion are in short supply. However, the paucity of health promotion evaluations is less disheartening when understood in the context of difficulties in undertaking and publishing evaluations, and when compared to the state of evaluation in other fields (Rootman et al., 2000).
Complexity in health promotion evaluation and its relation to evidence: developing our own set of ‘rules for evidence’
Recently, the powerful notion of complexity has entered into the evidence discussion. Health promotion is about change, e.g. change in communities or changes in policy. Much of health promotion is concerned with change in behavior; however, behaviors lie on a continuum of complexity. At one end are the individual behavioral changes and at the other end are changes in the behavior of a collective, e.g. a community. With the change of a single behavior in one individual there are many considerations, but relatively speaking these changes may be considered relatively simple. For example, an individual choosing to smoke or not smoke can be considered a dichotomous decision, with a strong emphasis on the individual's own choice to stop smoking. With changes that involve group or whole community changes in behaviors, the situation is much more complex. For example, the behaviors associated with the decisions of a community to build and invest in a mass transit system are in a complex web. Clearly, to create an intervention to address individual behavioral change or a single behavior in a restricted setting, when that variable relates to a single disease and the changing of the behavior will have only a single outcome, is less problematic than creating an intervention addressing many variables in the public at large with many possible outcomes.
The complexity that represents many, if not most community-based initiatives in health promotion affects profoundly measurement, design, analysis and the use of findings from such initiatives. The methods one needs to use and develop are tied to the complexity. As the complexity of an intervention increases we need more complicated methods of assessment; answers provided by the assessment may be less certain. It is also clear that the most rigorous method of assessment that many propose for evaluation, namely an RCT, is best suited to a simple intervention.
Gathering evidence for the value of health promotion remains a challenging task. However, we need to have a broad vision of evidence that embraces the inherent complexity of health promotion as a field. Rather than retreating to limited rules for what constitutes evidence, there is a need to look towards analytical frameworks that recognize the complexity of the field. It is a challenge for health promotion to convince its enthusiasts and detractors that there are no easy answers to complex human phenomena.
Summarizing some critical and unresolved issues around evidence
1. ‘Rules of evidence’ tied to disciplines, not projects
Over the years, scientific disciplines, e.g. physics, biology, epidemiology, etc. have developed their standards for what constitutes proof of causation, effect, etc. in observation and experiment. Thus, the appropriate ‘scientific method’ is both a product of historical development and the characteristic ‘observables’ in the discipline. There are differences among the disciplines for ‘rules of evidence.’ Many community-based public health prevention and health promotion projects are not discipline based, but represent a ‘field of action’. Therefore, there is no discipline-based epistemological structure underlying the evaluation of effort.
Action needed to address this issue:
- distinguish underlying rules of evidence for main disciplines of public health that are basic to health promotion, community-based research and intervention; and
- specify the role of epidemiology, social psychology, sociology, anthropology, health education and other relevant disciplines in evidence building for health promotion.
2. No consensus on any ‘hierarchy’ of evidence
Within the general area of community research, intervention and evaluation there is currently great debate about what constitutes knowledge in the field and what is evidence, or even whether the notion of evidence is applicable to the evaluation of interventions in communities. In summary, there is no consensus on any ‘hierarchy of evidence’ between researchers and practitioners in the field. International groups have asserted that it is premature to prioritize types of evidence in a linear hierarchy.
Action needed to address this issue:
- document where there is consensus and lack of consensus; consider pros and cons of consensus in the context of community prevention and suggest directions for the future; and
- synthesize for health promotion the findings and conclusions of the various national and international collaborations, task forces and working groups addressing evidence and evaluation.
3. Complexity of multi-disciplinary, compound interventions make simple universal rules of evidence untenable
Existing rules of evidence are often based on interventions that have relatively simple, demonstrable chains of causation, where single factors are manipulated to produce single easily measured outcomes. Many community-based health interventions include a complex mixture of many disciplines, many variables of varying degrees of measurement difficulty, and dynamic changing settings. In short, understanding multivariate fields of action requires a mixture of complex methodologies and considerable time to unravel any causal relationships. New analyses may reveal some critical outcomes in the years following an intervention.
Action needed to address this issue:
- recognize complexity as it pertains to community interventions and health promotion;
- suggest areas needing development to better understand analytical challenges;
- suggest more appropriate analytical methods and evaluation designs; and
- train the health promotion community in the use of appropriate methods.
Issues arising on evidence and health promotion at the Fifth Conference on Global Health Promotion in Mexico City, June 5–9, 2000
An ad hoc group of attendees from wide-ranging backgrounds met during the conference. This group, co-chaired by David McQueen and Ligia Salazar, was initiated by the World Health Organization (WHO) to help clarify and define the role of health promotion evaluation, and identify those gaps that need to be addressed. The outcome of these meetings was a set of recommendations for WHO to consider in health promotion evaluation.
There was clear recognition that while the evidence debate in the West has been prolific, from a global perspective many voices are still missing from the debate, notably those from developing countries. This lack of participation by those from developing countries is exacerbated by a debate that has been mainly conducted in the English language by those educated in a European–American context. Furthermore the debate has been largely by an academic elite or by those privileged to hold the type of government offices that allow them to come to the table. The workgroup felt that the debate must find a way to uncover these missing voices and the approaches used by developing nations that are meaningful. These voices and methods must be incorporated into the existing body of evidence. However, the mechanism for this remains very unclear and one can even question the legitimacy of such an elite at an international invited meeting positing an appropriate mechanism for inclusion.
The workgroup developed the following set of recommendations.
- WHO should establish a workgroup that will be responsible for creating a plan for the development of evaluation globally.
- This workgroup should build on work previously done by other workgroups and integrate unpublished work into current evaluation knowledge.
- Evaluation approaches should recognize the importance of equity in conducting locally determined evaluation, and should emphasize the use of participatory approaches and multisectoral involvement in evaluation.
- The workgroup should have equal representation from developed and developing countries and should have diverse cultural representation.
CONCLUSION
This paper only scratches the surface of what has become a considerable and complex debate. It is a debate that health promotion researchers and practitioners alike must fully embrace. Any field of action in public health rises above rhetoric only if it can ultimately demonstrate that it can make a difference to the public's health. Evaluation is at the heart of that demonstration. Producing evidence is only one key to sound evaluation.
Many in health promotion recognize that the field is at a critical juncture. There are calls to produce an evidence-based health promotion. Many traditional disciplines in public health have produced their standards for an evidence-based practice. Many of these standards have been adopted in health promotion practice. However, the field of health promotion is too broad to be limited by evidence standards derived from elsewhere. A challenge for the new century, for health promotion, is to foster and develop high quality, widely recognized and acceptable standards for evidence-based evaluation.
ACKNOWLEDGEMENTS
This paper was written initially as a technical report for the Fifth Global Conference on Health Promotion: Bridging the Equity Gap, Mexico, 5–9 June, 2000. Many thanks to the following and apologies to those inadvertently not mentioned: the CDC staff of Angel Roca, Melissa Fitz-Simons and Mary Hall; the members of the European Working Group on health promotion evaluation; the organizers of the 5th Global Conference on Health Promotion (i.e. WHO, PAHO, and the Ministry of Health, Mexico), particularly Ursel Broesskamp-Stone in Geneva, who initiated the conference ad hoc working group on evaluation; several members of the Guide Task Force and staff at CDC; the IUHPE Project Advisory Group members; and colleagues around the globe who have offered opinions, reflections and advice.
REFERENCES
Adrian, M., Layne, N. and Moreau, J. (1994) Can life expectancies be used to determine if health promotion works? American Journal of Health Promotion, 8, 449–461.[Medline]
Allison, K. and Rootman, I. (1996) Scientific rigor and community participation in health promotion research: are they compatible? Health Promotion International, 11, 333–340.
Bhaskar, R. (1997) A Realist Theory of Science. 2nd edn. Verso, New York.
Butcher, R. B. (1998) Foundations for evidence-based decision making. In Canada Health Action: Building on the Legacy, Vol. 5, pp. 259–290. Editions MultiMondes, Québec.
CDC (2000) Internet site for the Community Guide: http://www.health.gov/communityguide.
European Working Group on Health Promotion Evaluation (EWG) (1998) Health Promotion Evaluation: Recommendations to Policymakers. WHO (EURO), Health Canada, Ottawa.
International Union for Health Promotion and Education (IUHPE) (1999) The Evidence of Health Promotion Effectiveness: a report for the European Commission by the International Union for Health Promotion and Education. ECSC-EC-EAEC, Brussels–Luxembourg.
MacDonald, G., Veen, C. and Tones, K. (1996) Evidence for success in health promotion: suggestions for improvement. Health Education Research: Theory and Practice, 11, 367–376.
Nutbeam, D. (1998) Evaluating health promotion—progress, problems, and solutions. Health Promotion International, 13, 27–44.
Rootman, I., Goodstadt, M., McQueen, D., Potvin, L., Springett, J. and Ziglio, E. (eds) (2001) Evaluation in Health Promotion: Principles and Perspectives. WHO (EURO), Copenhagen.
Sackett, D. et al. (1996) Evidence-based medicine: What it is and what it isn't. British Medical Journal, 150, 1249–1255.
Suppe, F. (ed.) (1977) The Structure of Scientific Theories, 2nd edn. University of Illinois Press, Urbana, IL.
Supplement to American Journal of Preventive Medicine (SAJPM) (2000) Introducing the Guide to Community Preventive Services: Methods, First Recommendations and Expert Commentary. American Journal of Preventive Medicine, 18, 35–43.[ISI][Medline]
Tones, K. (1997) Beyond the randomized controlled trial: a case for A Judicial Review. Health Education Research, 12, 1–4.
Weiner, J. (1995) The Beak of the Finch: A Story of Evolution in Our Time. Random House, New York.
WHO, EURO (1984) Health Promotion: A Discussion Document on the Concepts and Principles. WHO, Copenhagen.
WHO, EURO (1998) Health Promotion Evaluation: Recommendations to Policymakers. WHO, Copenhagen.
World Health Assembly (1998) Resolution WHA 51.12 on Health Promotion. Agenda Item 20, 16 May 1998. WHO, Geneva.
CiteULike 
Connotea 
Del.icio.us What's this?
This article has been cited by other articles:
|
|
 |
|
  D. P. Behague and K. T. Storeng Collapsing the Vertical-Horizontal Divide: An Ethnographic Study of Evidence-Based Policymaking in Maternal Health Am J Public Health, April 1, 2008; 98(4): 644 - 649. [Abstract] [Full Text] [PDF]
|
|
|
|
 |
|
 B. Spencer, U. Broesskamp-Stone, B. Ruckstuhl, G. Ackermann, A. Spoerri, and B. Cloetta Modelling the results of health promotion activities in Switzerland: development of the Swiss Model for Outcome Classification in Health Promotion and Prevention Health Promot. Int., March 1, 2008; 23(1): 86 - 97. [Abstract] [Full Text] [PDF]
|
|
|
|
 |
|
 E. Waters, N. Priest, R. Armstrong, S. Oliver, P. Baker, D. McQueen, C. Summerbell, M. P. Kelly, and B. Swinburn The role of a Prospective Public Health Intervention Study Register in building public health evidence: proposal for content and use J. Public Health Med., September 1, 2007; 29(3): 322 - 327. [Abstract] [Full Text] [PDF]
|
|
|
|
 |
|
 L. Evans, M. Hall, C. M. Jones, and A. Neiman Did the Ottawa Charter play a role in the push to assess the effectiveness of health promotion? Promotion & Education, June 1, 2007; 14(2_suppl): 28 - 30. [Abstract] [PDF]
|
|
|
|
|
|
R. Bodstein The complexity of the discussion on effectiveness and evidence in health promotion practices Promotion & Education, March 1, 2007; 14(1_suppl): 16 - 20. [Abstract] [PDF]
|
|
|
|
 |
|
 S. K. Kumanyika and C. B. Morssink Bridging domains in efforts to reduce disparities in health and health care. Health Educ Behav, August 1, 2006; 33(4): 440 - 458. [Abstract] [PDF]
|
|
|
|
 |
|
 S. Audrey, J. Holliday, N. Parry-Langdon, and R. Campbell Meeting the challenges of implementing process evaluation within randomized controlled trials: the example of ASSIST (A Stop Smoking in Schools Trial) Health Educ. Res., July 1, 2006; 21(3): 366 - 377. [Abstract] [Full Text] [PDF]
|
|
|
|
 |
|
 N. B. Wallerstein and B. Duran Using Community-Based Participatory Research to Address Health Disparities Health Promot Pract, July 1, 2006; 7(3): 312 - 323. [Abstract] [PDF]
|
|
|
|
|
|
R. Armstrong, N. Jackson, J. Doyle, E. Waters, and F. Howes It's in your hands: the value of handsearching in conducting systematic reviews of public health interventions J. Public Health Med., December 1, 2005; 27(4): 388 - 391. [Abstract] [Full Text] [PDF]
|
|
|
|
 |
|
 M. A. Veazie, J. M. Galloway, D. Matson-Koffman, D. R. LaBarthe, J. N. Brownstein, M. Emr, E. Bolton, E. Freund Jr, R. Fulwood, J. Guyton-Krishnan, et al. Taking the Initiative: Implementing the American Heart Association Guide for Improving Cardiovascular Health at the Community Level: Healthy People 2010 Heart Disease and Stroke Partnership Community Guideline Implementation and Best Practices Workgroup Circulation, October 18, 2005; 112(16): 2538 - 2554. [Full Text] [PDF]
|
|
|
|
|
|
E. Lahtinen, P. Koskinen-Ollonqvist, P. Rouvinen-Wilenius, P. Tuominen, and M. B. Mittelmark The development of quality criteria for research: a Finnish approach Health Promot. Int., September 1, 2005; 20(3): 306 - 315. [Abstract] [Full Text] [PDF]
|
|
|
|
 |
|
 B. C K Choi, T. Pang, V. Lin, P. Puska, G. Sherman, M. Goddard, M. J Ackland, P. Sainsbury, S. Stachenko, H. Morrison, et al. Can scientists and policy makers work together? J. Epidemiol. Community Health, August 1, 2005; 59(8): 632 - 637. [Abstract] [Full Text] [PDF]
|
|
|
|
|
|
K. PATON, S. SENGUPTA, and L. HASSAN Settings, systems and organization development: the Healthy Living and Working Model Health Promot. Int., March 1, 2005; 20(1): 81 - 89. [Abstract] [Full Text] [PDF]
|
|
|
|
|
|
E. A. Abbema, P. Van Assema, G. J. Kok, E. De Leeuw, and N. K. De Vries Effect evaluation of a comprehensive community intervention aimed at reducing socioeconomic health inequalities in the Netherlands Health Promot. Int., June 1, 2004; 19(2): 141 - 156. [Abstract] [Full Text] [PDF]
|
|
|
|
 |
|
 J. Vinck, B. Oldenburg, and T. Von Lengerke Editorial: Health Psychology and Public Health--Bridging the Gap J Health Psychol, January 1, 2004; 9(1): 5 - 12. [PDF]
|
|
|
|
|
|
K C Tang, J P Ehsani, and D V McQueen Evidence based health promotion: recollections, reflections, and reconsiderations J. Epidemiol. Community Health, November 1, 2003; 57(11): 841 - 843. [Full Text] [PDF]
|
|
|
|
|
|
K. L. Philip, K. Backett-Milburn, S. Cunningham-Burley, and J. B. Davis Practising what we preach? A practical approach to bringing research, policy and practice together in relation to children and health inequalities Health Educ. Res., October 1, 2003; 18(5): 568 - 579. [Abstract] [Full Text] [PDF]
|
|
|
|
 |
|
 W. E. Thurston and L. Potvin Evaluability Assessment: A Tool for Incorporating Evaluation in Social Change Programmes Evaluation, October 1, 2003; 9(4): 453 - 469. [Abstract] [PDF]
|
|
| ||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||