Health Promotion International Advance Access originally published online on April 7, 2006
Health Promotion International 2006 21(2):145-152; doi:10.1093/heapro/dal006
| ||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||
Choosing indicators to evaluate Healthy Cities projects: a political task?*,
1 Quebec WHO Collaborating Center on the Development of Healthy Cities and Towns, Faculty of Nursing and Groupe de recherche et d'intervention en promotion de la santé (GRIPSUL), Université Laval Québec, Canada G1K 7P4 2 Quebec WHO Collaborating Center on the Development of Healthy Cities and Towns, Direction de la santé publique de l'Agence régionale de la santé et des services sociaux de l'Abitibi-Témiscamingue and Institut national de santé publique du Québec Québec, Canada
Address for correspondence: Michel O'Neill Quebec WHO Collaborating Center on the Development of Healthy Cities and Towns, Faculty of Nursing and Groupe de recherche et d'intervention en promotion de la santé (GRIPSUL), Université Laval, Québec, Canada G1K 7P4 E-mail: Michel.Oneill{at}fsi.ulaval.ca
 |
SUMMARY |
|---|
Ever since their beginning in 1986, Healthy Cities projects all over the world have been confronted with the issue of evaluation. However, after 20 years, many key dilemmas constantly reappear, people often looking for a kind of ‘magic’ list of universally applicable indicators to evaluate these initiatives. In this article we address five questions, allowing to illustrate the evaluative dilemmas the Healthy Communities movement is confronted with: Why evaluate Healthy Cities? What should be evaluated? Evaluate for who? Who should undertake the evaluation? How should the evaluation be performed? We conclude by formulating three recommendations in order to stimulate exchanges and debate. Our argument is based on a recent thorough analysis of the evaluative literature pertaining to the Healthy Cities movement, as well as on two decades of reflection on and involvement with this issue locally, nationally and internationally.
Key words: Healthy Cities; Healthy Communities; evaluation
|
EVALUATING HEALTHY CITIES: STILL AN ISSUE AFTER 20 YEARS |
|---|
 TOPSUMMARY EVALUATING HEALTHY CITIES: STILL... WHY EVALUATE HEALTHY CITIES?WHAT SHOULD BE EVALUATED?EVALUATE FOR WHO?WHO SHOULD UNDERTAKE THE...HOW SHOULD THE EVALUATION...CONCLUSION: THE NEED TO...REFERENCES |
|---|
The international Healthy Cities movement (HCM), since its beginnings in 1986, has been one of the flagship enterprises of the new vision of health promotion launched at that time by the World Health Organization (WHO); HCM was then seen as a privileged way to establish healthy public policy at the local level (Hancock and Duhl, 1988a
; Kickbusch, 1989; Evers et al., 1990; Ashton, 1991). However, the initiative outgrew the WHO context (Tsouros, 1992; Hancock, 1993) and became an international movement which currently has several thousand participating communities on all continents (Simard et al., 1997; Kenzer, 2000; Harpham et al., 2001; Izazola, 2004; Takano and Nakamura, 2004).
The issue of assessing whether or not this movement is making a difference has always been a key issue and finding the proper way to evaluate the variable and ‘fuzzy’ (Goumans, 1997) set of interventions the HCM has generated is still a matter of debate. However, this debate, in our viewpoint, has not advanced much because it fails to recognize that identifying the proper way to evaluate Healthy Communities initiatives is more than anything else a political task to be negotiated between various stakeholders. Hence, even today, we are still confronted all over the world with the dilemma of people longing for universal lists of indicators whereas in our viewpoint, it is obvious for many years that it is an improper way to approach the issue. Hopefully, by arguing as we do here the absolute necessity of taking into account the fundamentally political nature of the choices required to evaluate appropriately a Healthy City (HC) project, we will help many people confronted with this issue at the local, national or international levels to achieve more enlightened decisions faster.
We will do so by addressing first five questions: Why evaluate Healthy Cities? What should be evaluated? Evaluate for who? Who should undertake the evaluation? How should the evaluation be performed? Our argument is based on a recent thorough analysis of the evaluative literature pertaining to the HCM (Simard, 2005), as well as on two decades of reflection on and involvement with this issue locally, nationally and internationally.
|
WHY EVALUATE HEALTHY CITIES? |
|---|
Why should we evaluate an initiative like HCM? Evaluation specialists name several reasons, increasingly focused on the capacity of stakeholders to find something useful and relevant in these operations (Patton, 1997
; Lincoln and Guba, 2000). In our experience, five stand out with HCM. The first and usually the predominant one is to assess whether the movement has changed anything in municipal political processes, in the health of the community or in any other characteristic. If membership in the movement offers no benefit, or does not deliver what its promoters hoped for, evaluation would normally encourage changes to the project or activities to achieve better results. A second reason is to maintain the political legitimacy of a Healthy Cities project (HCP). A third is comparing oneself with others; in our experience all over the world, politicians are usually very eager to do so and often citizen groups or community organizations as well. A fourth has to do with community mobilization and sustainability; demonstrating the success achieved constitutes a good method to ensure further participation as well as more sustained commitment to organizing other activities. A fifth and more academic one is to contribute to scientific knowledge.
What we wish to emphasize here is that all the reasons above to evaluate HCM are legitimate, even if of very different nature, and for us none is automatically more valuable than the other.
|
WHAT SHOULD BE EVALUATED? |
|---|
Several aspects can be taken into consideration when identifying elements to be evaluated within the broad universe of HCM (Davies and Kelly, 1993
; Poland, 1996a,b; Costongs and Springett, 1997a; Cherbonnier, 1998; Kegler et al., 2000; Dowbor, 2001). In the following figure, we propose a framework to organize them according to level (from an activity in a local initiative or project to the international movement and its networks as a whole) and types; it defines a set of areas, out of which a whole research programme could be derived, and helps to position various possible evaluation enterprises. These areas can be considered both from the internal viewpoint of the stakeholders at a specific level or from the external viewpoint of a funding agency, a government or other institutions outside of the level analysed (Figure 1). Our discussion on what to evaluate is based on this framework.
|
Creating health profiles to evaluate the health status of a community
When one says that a community is healthy, what does it exactly mean? This is a key question because one of the first suggestions to any locality willing to get involved into HCM is to produce its health profile as the basis for a health plan (WHO-Euro, 1992
; Costongs and Springett, 1997b). However, this is a major challenge.
First, given the value base of HCM, it seems central to address the health status of the community as an entity and not just as a set of aggregated individuals (Hancock et al., 2000). What unit should be utilized to measure the health status of a community then? A sentence in a brochure for the Vivre Montréal en santé programme (VMS, 1990) gives in a nutshell a very good idea of the issue: creating a Healthy Montréal means taking action as a community to improve the quality of life for every sector of the city, every neighborhood in each sector, every street in each neighborhood, every home on each street and every individual in each home. So, which one should be chosen when all these levels make sense in some way?
Second, beyond the unit of measurement, the conceptual issues of defining health need to be carefully considered. Given the vision espoused by HCM which goes beyond a mere absence of illness to address well-being and quality of life, it becomes a major challenge, especially with regard to the positive aspects of health for communities as entities (Hancock et al., 2000). Various dimensions of community functioning such as democratic life, social cohesion, community capacity or social capital can then be measured, which is a totally different enterprise than if aggregated individualized data on mortality or morbidity are utilized as is still often the case.
These issues are well exemplified by the evolution of the oldest and most sophisticated evaluative HCM undertaking, the European one. In its early Guide to Assessing Healthy Cities (Hancock and Duhl, 1988b), WHO-Euro proposed a set of categories to evaluate whether a city is healthy including geography, history, demographics, political structures, economics, social problems, religion and a sense of belonging to the city; however, no specific way to measure them was suggested. The original attempt made to assess the success of the initiative, after the first 5 years of operation, was much more precise; it proposed to cities in the project to utilize the list of 219 indicators developed to measure on a national scale the progress towards the 38 goals of Health For All in Europe (WHO, 1989). As that strategy proved impossible to implement, at the start of the 1990s and piloted by the city of Nancy in France, another one was devised by the various cities that were then in the project, using 60 indicators (3 on health, 11 on health services, 19 on environment, 20 on social and economic concerns and 7 on general information). Its utilization proved problematic as well and many issues for evaluating this initiative in Europe still remain unsolved after almost 20 years of operation (Doyle et al., 1997; Curtice, 2001; de Leeuw, 2001).
In fact, there is already an incredibly large number of indicators that can be utilized as a result of a broad definition of health in communities, from classical epidemiological indicators to whole sets of environmental, social or economic ones. This has led to a great number of lists, each containing a variable number of very legitimate indicators (Baum and Brown, 1989; Cappon, 1989; Noack, 1989; Cardinal and Pageau, 1990; Feather and Mathur, 1990; Hayes and Willms, 1990; Stevenson and Burke, 1990; VMS, 1990; Cappon, 1991; Craig et al., 1991; Garretsen et al., 1991; Flynn, 1992; Chevalier and Fortin, 1995; OPS, 1996; Werna et al., 1998; Curtice et al., 2001; Harpham et al., 2001); and many more could be devised just using what is already available. But how then to choose among them to develop the health profile of a precise locality?
Evaluating the process of introducing HCM into a community
A second set of elements that can be evaluated is to take a look at how HCM is introduced into a specific community or entity. Who was involved? What challenges were faced? What strategies were utilized? What was achieved in terms of evolution of the policy arena? This has been rarely done, however (Fortin et al., 1992; Manson-Singer, 1994; Nunez et al., 1994; Ouellet et al., 1994; Werna, 1995; Goumans and Springett, 1997; Boonekamp et al., 1999; Burton, 1999; Adams, 2000; Barten, 2000), even if it may provide key strategic indications for those wanting to launch a HC project in addition to useful scientific knowledge for public policies analysts.
The evolution of local processes can also be analysed from the point of view of community change, of empowerment. It is currently a very important tendency in the evaluation literature (Fetterman et al., 1996; Minkler and Wallerstein, 2003; Simard, 2005), totally in line with some of the recent and most promising HCM evaluative undertakings: several tools and guidebooks have already been developed to empower communities to self-evaluate their HC initiative (Speller and Funnell, undated; B.C. Ministry of Health, 1991; Ville de Montréal, 1993; Maltrud et al., 1997; Tyler Norris Associates, 1997; Guidry, 2001; PAHO, 2003; Wallerstein et al., 2003).
Evaluating the impact of a HCP
Even though the rhetoric of health promotion often strongly insists on processes, evaluating the impacts of HCM will become absolutely necessary at some point in time (O'Neill, 1991; Goumans, 1997). Given the worldwide emphasis on evidence-based decision making which has also invaded health promotion (O'Neill, 2003), the issue of how to define impact is all the more crucial in order to take into consideration on the one hand intermediate outcomes (Nutbeam, 1998) and on the other hand a wider variety of results than just mortality or morbidity (Hancock et al., 2000). Sooner or later, every HC project will need to go beyond a first profile of its health status and the analysis of its processes to assess if, in fact, its undertakings have produced any change at all, no matter how defined and measured. The current work about the effectiveness of health promotion community interventions provides in this respect interesting directions (Hills et al., 2004).
|
EVALUATE FOR WHO? |
|---|
If we consider for whom the evaluation is produced, there are highly diverse clienteles with varied expectations, here again all legitimate.
The various implementers of a HC initiative, be they civil servants, public health professionals, other professionals or community members, are often the most eager users of evaluation results. For them, the required information is usually of an administrative nature, in order to monitor the evolution of the initiative or to reflect on their practice.
Evaluation may also be performed for the needs of elected municipal authorities, generally interested in knowing whether funds have been properly used or whether the results are popular with the electorate. Others interested may as well be national or international bodies providing financing or legitimacy to the initiative.
If the main client of the evaluation is the general public then participatory (Minkler, 2000), fourth-generation (Guba and Lincoln, 1989) or utilization focused (Patton, 1997) approaches can become very useful, as well as a process to disseminate the results which is appropriate to reach that group. It is in the communities' interest to evaluate their projects in order to see the progress made and, following the health promotion commitment towards empowerment, there is currently a strong trend towards self-evaluation approaches as mentioned earlier.
Finally, the main clientele for an academic researcher, given the rewards system that makes it possible to obtain and maintain its position, is generally the local, national or preferably the international scientific community. This obliges to disseminate results mostly in English, even for the non-native English speaker, and mainly through peer reviewed conferences and publications generally inaccessible to almost everybody but academics.
Evaluation matters, whoever it is performed for. However, those financing the project most often are the ones who will shape the type of evaluation performed and evaluation always means potential control and sanction, even if this is often denied. Who is the client for the evaluation is thus far from being a simple intellectual question; it will usually determine the very nature of the process and, in the end, the nature of the knowledge produced and its utilization for decision-making.
|
WHO SHOULD UNDERTAKE THE EVALUATION? |
|---|
Depending on who does the evaluation work, the approach can be very diverse. If academics are doing it, the evaluation will usually be properly designed according to scientific standards but will probably take several months or even years to complete, longer than most users are usually comfortable to wait.
If the evaluation is done by professional researchers or specialists from local, national or international public agencies the evaluation will likely have more of an administrative or bureaucratic intent allowing to make adjustments to improve operations.
If it is the general population that is requested to evaluate the Healthy Cities initiative, as mentioned earlier, numerous methodologies for self-evaluation have been developed to assist communities in doing so. The electoral process is another form of evaluation by the population of a political team which has chosen (or refused) to include HC in its activities. Other forms of evaluation by the population such as opinion surveys can also be informative on the degree of satisfaction with the project.
As for the previous questions it is evident, and once again this is totally legitimate in our viewpoint, that the evaluation will inevitably be influenced by the individuals or institutions designing and conducting it. Two major trends can be identified in this respect: evaluations initiated or undertaken from outside the community, which can be labeled as external or exogenous, and those originating from within the community and often completed as self-evaluations, which can be qualified of internal or endogenous; both are useful and necessary, even if they lead to different types of results. In many places, however, academic researchers, practitioners, communities and policy-makers have reached interesting agreements concerning evaluation of local development initiatives, in which joint efforts external and internal to the community are made that are relevant for action, useful for the community as well as scientifically relevant for the advancement of knowledge (Hills and Mullett, 2000; Minkler and Wallerstein, 2003). It is probably from such integrated processes that the most interesting developments will emerge in the future.
|
HOW SHOULD THE EVALUATION BE REALIZED? |
|---|
The manner in which the four previous questions are answered will determine which ones, in the wide range of methodological and technical possibilities, will be retained to evaluate a specific HC initiative. It is easy to fall into epistemological or methodological debates so dear to academics such as: Qualitative or quantitative approaches? Should new data be collected (a costly effort) or should existing data be analysed in agreement with the project needs? Should researchers be concerned with seeing their results integrated into the decision-making process or leave that work to others? etc. The how of an evaluation offers just as wide a range of issues as the other questions, all equally legitimate and interesting.
In our viewpoint, it is nevertheless important to emphasize the need for evaluation methods to be increasingly in accordance with the values on which HCM is founded, namely participatory and intersectoral approaches as very well advocated for in the work of Hancock et al. (Hancock et al., 2000).
|
CONCLUSION: THE NEED TO NEGOTIATE PRAGMATIC CHOICES |
|---|
How then, in an actual situation, should one react when confronted to the wide range of answers provided to each one of the five previous questions? Why is the choice of indicators and evaluation strategies for HCM a process so full of difficulties? What conclusions do we reach at the end of our analysis? The dilemmas raised by the questions addressed above are difficult and there are no miracle solutions.
As it is always the case, various groups and individuals, with diverse viewpoints and varying levels of authority to impose them, will inevitably have opinions that can dramatically differ in what, how, why, for whom and by whom the evaluation should be done. What we have observed in the HCM context, however, is that it generally leads to an unusually extended intersectoral and participatory approach. As a result, there are almost systematically more players involved in HCM than in most other public health initiatives, leading to a greater probability of dissension and misunderstanding over evaluation goals and processes. At the same time, it shows the relevance of perspectives like utilization focused evaluation or empowerment evaluation as ways to go about doing such evaluations.
This leads us to three conclusions. First, it appears absolutely essential to us, despite the obstacles, to evaluate HC projects in order to allow those involved in the opportunity to make a critical examination of their work as well as to demonstrate to funding bodies that results justifying their investments are reached. The need for evaluation can always be put off until a later time but inevitably it will become necessary at some point. Our second conclusion is to strongly reject a uniform and monolithic approach to evaluation. There is no magic list of indicators that can universally be used and can be applied to any HC project in the world, as there is no unique ‘better’ way of doing HC evaluation. Tension will always exist between groups or individuals on the priorities for evaluation and on the methods to achieve them.
Our final and most important conclusion is thus the need to make negotiated choices, which are political choices by necessity since there are almost always conflicting goals and interests at stake. As we have emphasized, there are a vast number of possible and legitimate ways to select indicators or evaluation strategies. Each HC project must thus decide for itself its short, mid and long-term needs for evaluation and equip itself with an evaluation process meeting its own needs, even if it means that it is more difficult to compare with others. We finally believe, for having seen it more than once, that if properly done, such a negotiation process and the reaching of a consensus on evaluation among the various stakeholders will probably be as important for developing a project as the results of the evaluation itself.
As final words, we are conscious that rather than taking a reflexive stand on these issues out of our experience, we could have done a different type of job by using evaluation frameworks or political science theories; many others in the reference list we provide have done so, and very aptly in many cases. We obviously do not deny either that there is a paucity of empirical evaluative research on HCM and that more is always needed. What we wanted to insist on here though, to stimulate reflection and debate, is some of the recurring practical issues we see reemerging over and over, despite all the good work done!
|
FOOTNOTES |
|---|
*A preliminary version of this article was presented at the workshop Community level indicators: building community capacity for health in Jasper, Canada, October 2002. Thanks to the organizers who provided the resources for a translation from French of that paper. Thanks as well for the comments of two anonymous reviewers that were very useful in the revision of this manuscript.
A few semantic precisions. Internationally, the tendency is to talk about the Healthy Cities movement, whereas in Latin-America the expression Municipios saludables (Healthy Municipalities) is utilized. In many countries as in Canada for instance, as most communities are much smaller than cities, the choice has been made to talk about the Healthy Communities movement. In the Pacific Island countries, it is the expression Healthy Islands which has been retained due to the settlement patterns there. Finally, the original WHO Healthy Cities project was clearly aimed at local municipal authorities, whereas in many places in the world the movement does not systematically involve municipal governments. In this article, we will thus use the expression Healthy Cities to cover these different realities and the word local rather than municipal to reflect the frequent reality of HCP implementation without City Hall.
|
REFERENCES |
|---|
Adams C. (2000) Healthy Communities and public policy: four success stories. Public Health Reports 115:212–215.[CrossRef][Web of Science][Medline]
Ashton J. (1991) The Healthy Cities Project: a challenge for health education. Health Education Quarterly 18:39–48.[Web of Science][Medline]
B.C. Ministry of Health. (1991) Health Indicator Workbook: A Tool for Healthy Communities (British Columbia Ministry of Health, Office of Health Promotion, Victoria).
Barten F. (2000) Managua Healthy Municipality Initiative—Final Evaluation Report (UNDP-LIFE/WHO-PAHO, Washington).
Baum F. and Brown V. (1989) Healthy Cities (Australia) project: issues of evaluation for the new public health. Community Health Studies 13:140–149.[Web of Science][Medline]
Boonekamp G., Colomber C., Tomas A., Nunez A. (1999) Healthy Cities evaluation: the co-ordinators perspective. Health Promotion International 14:103–109.
Burton S. (1999) Evaluation of healthy city projects: stakeholder analysis of two projects in Bangladesh. Environment and Urbanization 11:41–53.[Abstract]
Cappon D. (1989) Indicators for a healthy city. Environmental Management and Health, An International Journal 1:113–122.
Cappon D. (1991) Criteria for healthy urban environments. Canadian Journal of Public Health 82:249–258.[Web of Science][Medline]
Indicateurs sanitaires, sociaux et environnementaux pour une municipalité en santé: un cadre opérationnel Cardinal L. and Pageau M. (1990) Groupe de travail francophone de l'OMS sur les indicateurs pour les Cités-santé, Nancy.
Question Santé, Cherbonnier A. (1998) Promouvoir la santé dans la ville. Réflexions autour de la concertation, la participation, l'évaluation dans le cadre du concept "Ville-Santé". 6:3–8.
Chevalier S. and Fortin L. (1995) Indicateurs sociaux-économiques: définitions et interprétations (Institut canadien d'information sur la santé, Ottawa).
Costongs C. and Springett J. (1997a) Towards a framework for the evaluation of health-related policies in cities. Evaluation 3:345–362.
Costongs C. and Springett J. (1997b) Joint working and the production of a City Health Plan: the Liverpool experience. Health Promotion International 12:9–19.
Craig N., et al. (1991) Information for planning Healthy Communities. Proceedings of an invitational workshop (Center for Health Promotion, University of Toronto, Toronto).
Curtice L. (2001) Evaluation in urban settings: the challenge of Healthy Cities. In Rootman I. (Ed.), et al. Evaluation in Health Promotion. Principles and Perspectives WHO Europe, WHO Regional Publications European Series pp. 309–337.
Davies J. and Kelly M. (1993) Healthy Cities: Research and Practice. , Routledge, London and New York.
de Leeuw E. (2001) Global and local (glocal) health: the WHO healthy cities programme. Global Change and Human Health 2:34–45.
Dowbor T. (2001) Barriers and Facilitators for the Evaluation of Local Health Municipality Initiatives in Brazil: Contextual, Methodological and Practical Issues (Department of Public Health Sciences, Masters Thesis, University of Toronto, Toronto).
Doyle Y., Webster W., Price C. (1997) Healthy Cities Indicators: Analysis of Data from Cities Across Europe (WHO, Regional Office for Europe, Copenhagen).
Evers A., Farrant W., Trojan A. (1990) Healthy Public Policy at the Local Level (Campus Westview, Boulder).
Feather J. and Mathur B. (1990) Indicators for Healthy Communities. Proceedings of a National Workshop (Department of Community Health and Epidemiology, University of Saskatchewan, Institute of Urban Studies, University of Winnipeg, Winnipeg, Manitoba).
Fetterman D. M., Kaftarian S. J., Wandersman A. (1996) Empowerment Evaluation. Knowledge and Tools for Self-Assessment and Accountability (Sage Publications, Thousands Oaks/London/New Delhi).
Flynn P. (1992) Measuring health in cities. In Ashton J. (Ed.). Healthy Cities (Open University Press, Milton Keynes, Philadelphia).
Fortin J.-P., et al. (1992) Quebec's Healthy Communities projects, the ingredients of success. Health Promotion 31:6–10.
Garretsen H. F. L., et al. (1991) Collecting health information at a local level. Health Promotion International 6:121–133.
Goumans M. (1997) Innovations in a Fuzzy Domain: Healthy Cities and (Health) Policy Development in The Netherlands and the United Kingdom. (University of Maastricht, Maastricht) Faculty of Health Science, Ph.D. Thesis.
Goumans M. and Springett J. (1997) From projects to policy: "Healthy Cities" as a mechanism for policy change for health? Health Promotion International 12:311–322.
Guba E. G. and Lincoln Y. S. (1989) Fourth Generation Evaluation (Sage Publications, Newbury Park).
Guidry M. (2001) Healthy People in Healthy Communities. A Community Planning Guide Using Healthy People 2010 (Office of Public Health and Science, Department of Health and Human Services, Washington DC).
Hancock T. (1993) The evolution, impact and significance of the Healthy Cities/Healthy Communities Movement. Journal of Public Health Policy 1:5–18.[Medline]
Hancock T. and Duhl L. (1988a) Promoting Health in the Urban Context (FADL Publishers, Copenhagen).
Hancock T. and Duhl L. (1988b) WHO Healthy Cities project: A guide to assessing Healthy Cities (FADL Publishers, Copenhagen).
Hancock T., Labonté R., Edwards R. (2000) Indicators that Count! Measuring Population Health at the Community Level (Centre for Health Promotion/ParticipACTION, Toronto).
Harpham T., Burton S., Blue I. (2001) Healthy city projects in developing countries: the first evaluation. Health Promotion International 16:111–125.
Hayes M. and Willms S. (1990) Healthy community indicators: the peril of the search, the paucity of the find. Health Promotion International 5:161–167.
Hills M. and Mullett J. (2000) Methodologies and Methods for Community-based Research and Evaluation (Community Health Promotion Coalition, University of Victoria, Victoria, BC).
Hills M., Carroll S., O'Neill M. (2004) Vers un modèle d'évaluation de l'efficacité des interventions communautaires en promotion de la santé: compte-rendu de quelques développements Nord-américains récents. Promotion and Education S1:17–21.
Izazola H. (2004) Healthy cities in developing countries: lessons to be learned. International Development Planning Review 26:111–112.
Kegler M. C., Twiss J. M., Look V. (2000) Assessing community change at multiple levels: the genesis of an evaluation framework for the California Healthy Cities Project. Health Education and Behavior 27:760–779.
Kenzer M. (2000) Healthy Cities: a guide to the literature. Public Health Reports 115:279–289.[CrossRef][Web of Science]
Kickbusch I. (1989) Healthy Cities: a working project and a growing movement. Health Promotion 4:77–82.
Lincoln Y. S. and Guba E. G. (2000) Paradigmatic Controversies, Contradictions and Emerging Confluences. In Lincoln Y. S. (Ed.). Handbook of Qualitative Research 2nd edition (Sage Publications, London).
Maltrud K., Polacsek M., Wallerstein N. (1997) Participatory Evaluation Workbook for Community Initiatives. , New Mexico.
Manson-Singer S. (1994) The Canadian Healthy Communities Project: creating a social movement. In Pederson A. (Ed.), et al. Health Promotion in Canada (W.B. Saunders, Toronto) pp. 107–122.
Minkler M. (2000) Using participatory action research to build healthy communities. Public Health Reports 115:191–197.[CrossRef][Web of Science][Medline]
Minkler M. and Wallerstein N. (2003) Community-based Participatory Research for Health (Jossey-Bass, San Francisco).
Noack H. (1989) Paper prepared for the annual Healthy Cities symposiumPécs, HungaryHealth Information for Healthy Cities.
Nunez A., Colomer C., Peiro R., Alvarez-Dardet C. (1994) The Valencian community healthy cities network: assessment of the implementation process. Health Promotion International 9:189–198.
Nutbeam D. (1998) Evaluating health promotion—progress, problems and solutions. Health Promotion International 13:27–44.
O'Neill M. (1991) Evaluating Healthy Cities: the most urgent research task. Research for Healthy Cities 2:1–3.
O'Neill M. (2001) Les enjeux de l'évaluation des projets de développement social et des communautés (Colloque de l'ASPQ "Le développement social ou des communautés: acteurs ou spectateurs", Journées annuelles de santé publique, Montréal).
O'Neill M. (2003) Pourquoi se préoccupe-t-on tant des données probantes en promotion de la santé? SPM International Journal of Public Health 48:317–326.
OPS. (1996) Municipios saludables en América Latina: indicadores de salud y bienestar. Boletin de la Oficina Sanitaria Panaméricana 120:155–161.
Ouellet F., Durand D., Forget G. (1994) Preliminary results of an evaluation of three Healthy Cities initiatives in the Montreal area. Health Promotion International 9:153–159.
PAHO. (2003) Participatory Evaluation of Healthy Municipalities: A Practical Resource Kit for Action (Draft) (Pan American Health Organisation, Washington DC).
Patton Michael Quinn. (1997) Utlization-Focused Evaluation (Sage Publications, Thousands Oaks/London/New Delhi).
Poland B. D. (1996a) Knowledge development and evaluation in, of and for Healthy Community initiatives. Part 1: guiding principles. Health Promotion International 11:237–247.
Poland B. D. (1996b) Knowledge development and evaluation in, of and for Healthy Community initiatives. Part II: potential content foci. Health Promotion International 11:341–349.
Simard P. (2005) Perspectives pour une évaluation participative des Villes et Villages en santé. (Institut national de santé publique du Québec, Rouyn-Noranda).
Simard P., O'Neill M., Ousseynou E. D., El Housseynou L., Djénèba D., Halidou B. (1997) Contraintes et opportunités pour la mise en oeuvre du mouvement Villes et villages en santé en Afrique francophone (rapport synthèse) GRIPSUL, IAGU et RQVVS, Québec et Dakar.
Speller V. and Funnell R. (undated) Towards Evaluating Healthy Alliances (Institute of Public Health/Health Education Authority, Wessex).
Stevenson H. and Burke M. (1990) Selected Healthy City Indicators: A Research Agenda (York University Center for Health Studies, Toronto) pp. 72.
Takano T. and Nakamura K. (2004) Participatory research to enhance vision sharing for Healthy Town initiatives in Japan. Health Promotion International 19:299–307.
Tsouros A. (1992) World Health Organization Healthy Cities Project: A Project Becomes a Movement, Review of Progress, 1987 to 1990. Published for the WHO Healthy Cities Project Office (FADL Publishers, Copenhagen).
Tyler Norris Associates. (1997) The community Indicators Handbook. Measuring Progress Toward Healthy and Sustainable Communities. Tyler Norris Associates, Seattle.
Ville de Montréal. (1993) Guide pour un portrait de quartier Vivre Montréal en santé, Montréal.
VMS. Vivre Montréal en santé c'est agir ensemble... pour améliorer la qualité de vieVille de Montréal, Montréal.
Wallerstein N., Rice M., Akermann M., Cerqueira M. T., Hills M., Mendes R., et al. (2003) SOPHE Midyear Scientific Conference. , Las Cruces.
Werna E. (1995) The implementation of the Healthy Cities project in developing countries: lessons from Chittagong. Habitat International 19:1–8.[CrossRef]
Werna E., Harpham T., Blue I., Goldstein G. (1998) Evaluating Healthy City projects. In Werna E., Harpham T., Blue I., Goldstein G. (Eds.). Healthy City Projects in Developing Countries: An International Approach to Local Problems , Earthscan pp. 86–103.
WHO. (1989) Revised List of Indicators and Procedure for Monitoring Progress Towards Health for All in the European Region (1987–1988) (WHO European Bureau, Copenhagen).
WHO-Euro. (1992) Twenty Steps for Developing a Healthy Cities Project (World Health Organization, regional office for Europe, Copenhagen).
CiteULike 
Connotea 
Del.icio.us What's this?
| ||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||