Health Promotion International, Vol. 16, No. 2, 187-195,
June 2001
© Oxford University Press 2001
Quality indicators for health promotion programmes
Department of Public Health Sciences, Karolinska Institutet, Sundbyberg, Sweden, 1 Federation of Swedish County Councils, Stockholm, Sweden, 2 Department of Public Health Sciences, Karolinska Institutet, Solna, Sweden, 3 Unit of Public Health at Sörmland County Council, Sweden and 4 Stockholm County Centre of Public Health, Stockholm, Sweden
Address for correspondence: Marianne Granath, Landstinget Sörmland, Folkhälsoenheten, 611 88 Nyköping, Sweden, E-mail: marianne.granath{at}lk.dll.se
SUMMARY
Methods for systematically following up and auditing health promotion have been in demand for a considerable period of time. Quality assurance as an auditing method has opened up new opportunities in this area. On the basis of Donabedian's ‘triad’ of structure, process and outcome, the theoretical preconditions for and implementation of a number of successful health promotion programmes/ projects have been analysed with regard to their common characteristics. These characteristics have been generalized and then transformed into indicators of a successful health promotion programme/project. To ensure the practical applicability of the quality indicators, they were operationalized in what we call a ‘question pro-forma’. Any negative response to a question on the pro-forma indicates quality defects in a programme, and any positive response the opposite. The ‘template’ can be employed for both the planning and auditing for quality assurance on health promotion programmes and projects. The question pro-forma has been tested successfully on a number of programmes and projects. The results from one study are shown in the article.
Key words: health promotion programme; health promotion project; quality assurance; quality indicators
INTRODUCTION
Target 31 of the WHO Health for All Strategy states that ‘by 1990 all European states should have effective mechanisms for ensuring quality of patient care’(WHO, 1983). Quality of care has been an issue in national health services in many countries (Fry, 1990
; Landstingsforbundet, 1994). Total quality management (TQM) has been the guiding principle for many of these efforts, especially those concerning service provision (Cullen and Hollingum, 1987). The introduction of purchasing authorities, whose role it is to place contracts for health care, has highlighted the need for specification of work programmes and quality. Health promotion practice has increasingly become subject to similar demands of defining and assessing activities and effects.
The purpose of this article is to share the results of a review made to establish important aspects of successful health promotion projects. The purpose is also to demonstrate how these aspects have been transformed into indicators and a question pro-forma/a template for quality assurance of health promotion projects.
Quality assurance as a development tool in health promotion
Studies show that health promotion work can be both human-welfare promoting and a cost-effective way of handling health problems, but there has been a lack of methods for systematically following up and auditing health promotion programmes and projects. Health promotion is a long-term strategy, and its outcomes cannot often be confirmed before a lengthy period of time has elapsed. There is a need for methods of intermediate assessment to develop the work of programmes and projects as well as increasing effectiveness.
Besides being the answer to purchaser demands, some authors maintain that quality assurance could provide an opportunity for development and audit activities within the arena of health promotion (Rosén, 1992).
The concept of quality assurance
The concept of ‘quality assurance’ encompasses methods for describing, measuring, evaluating and, where needed, taking measures aimed at the improvement of what, in a broad sense, is described as quality (Calltorp and Bergström, 1992). Quality assurance refers to the work that takes place within any work unit, so as to follow-up and improve the unit's own activities and to prevent mistakes or defects from arising. Quality assurance differs from evaluation in that it focuses on ongoing work rather than any final outcome at a given point in time. Furthermore, quality assurance is often described as a continuous and dynamic process (Neuhauser et al., 1995).
How to establish quality indicators
Quality assurance is conducted by defining key areas of importance. A quality indicator is a key concept in the context of quality assurance, to which we employ the following definition: a specially selected measure or attribute that may indicate and point to good or poor quality. Establishing sets of indicators for health promotion would provide practitioners with the tools for systematically conducting the intermediate follow-ups that are currently lacking. It would help to pinpoint and articulate areas of development for projects and programmes, thereby increasing possibilities for goal attainment (Macdonald, 1992). Quality assurance would also provide a basis for dialogue with purchasers on providing adequate funding for creating and maintaining quality performance; something which would probably increase the quality of work and long-term effects of health promotion, often suffering under the myth that this type of work can be done by volunteers with their bare hands.
LITERATURE REVIEW
Within the quality assurance field, as in the evaluation arena, it is common to take a point of departure in what is called Donabedian's ‘triad’ (Donabedian, 1996) of structure, process and outcome.
A meta-study was conducted by means of examining textbooks, manuals and previous investigations (Key et al., 1976; Green and Lewis, 1986; Johnston, 1988; Bracht, 1990; Nutbeam et al., 1990), and also project reports, evaluation proposals and evaluations (Puska et al., 1985; Sandersson and Svanstrom, 1985; Altman, 1986; Schelp, 1987), to search for elements that have proved to be necessary and important in health promotion. We then generalized these into ‘sensitizing concepts’ (Cohen, 1970), which can act as indicators for health promotion programmes and projects. These are subsumed under the three headings of structure, process and outcome.
In this way we obtained a list of indicators, which can be regarded as jointly providing ‘a least common denominator’ of quality assurance in health promotion. The indicators are operationalized in a number of general questions per indicator. The final result is what we call a ‘question pro-forma’, within the confines of which any positive response to a question indicates favourable quality, and any negative response unfavourable quality. The questions could be adapted to different subject areas/types of programmes. The result of one test using the question pro-forma is presented below.
CONCEPTUAL THEMES AND QUALITY INDICATORS AS RESULTS FROM THE LITERATURE REVIEW
The result of the literature review is summarized below, describing the content of each indicator. The indicators have been operationalized into a question pro-forma. Some examples are demonstrated in the figures below.
Structure
Several authors have suggested that it is important for certain community-related and organizational conditions to be met if the implementation of a health promotion programme/ project is to be feasible (Johnston, 1988; Berwick et al., 1990; Bjaras and Haglund, 1990; Bracht, 1990; Haglund, 1990; Rosén, 1992). These authors take up the importance of problem description, goal formulation, the definition of target groups, and adequate decision-making, resources and work organization. These aspects correspond to the indicators we have chosen.
Goals
A key to success seems to lie in clear and specific problem description. Primary prevention is long-term by nature. The outcome can often not be detected before a lengthy period of time has elapsed. For this reason, it is even more important to formulate unequivocal and measurable goals, both of an overall nature and those that apply to structure, process and outcome. Up to the time that the actual results of a programme become apparent, there is every reason to use a variety of indicators of intended effects as a means of control.
One of the most important tasks in health promotion is to reduce health inequalities between different groups in society. For this reason, it is important to consider aspects of equality in the course of goal formulation.
Target groups
Performing a community analysis and/or a risk analysis provides the conditions under which programmes can be targeted at precisely those parts of the population affected by the problem in question. Therefore, the capacity to get the message across increases, and also the programme's cost-effectiveness. For the same reasons, opportunities for effective follow-up of the programme are improved. It is likewise important to recognize other reasons for starting projects.
Design
A theoretically rooted design and a model for programme strategy and activities constitute a further important factor (Sanderson and Svanstrom, 1985; Schalma and Kok, 1995). In this context, ethical aspects of the programme should also be considered.
Responsibility
An important condition for a successful programme seems to be that decisions can be reached at the level within the organization where resources are allocated (Fisker, 1992). As in all organizational contexts, it is important that there is a clear distribution of responsibility and authority.
Resources
Resources can be of both a material and human nature. The relationship between resources available and level of ambition is reasonable. It is also of importance that a programme's management can itself allocate and take decisions over resources so as to be able to adapt them according to need. The skills level of personnel is another key question, as too are opportunities to follow developments within their area by means of education and supervision.
Organization
Clear organization and defined leadership seems to constitute an important factor for successful operation. Any organizational description should therefore include a specification of the mandated tasks and authorities of personnel within a programme. The relationship between the programme's needs and its organization must also be reasonable. Among other things, it is important that the people in charge of a programme occupy a sufficiently independent position for them to act in relation to any partner collaborating on the programme. Finally, the programme's schedule must be adapted to what is expected in terms of results, and so on.
Process
Our meta-study obtained meagre results with regard to the description of the process of health promotion. One exception lies in a paper of Nutbeam et al., who prepared an extensive inventory of researchable areas within the process of health promotion, in particular in the field of tobacco consumption (Nutbeam et al., 1990). The authors found virtually no studies that evaluate or demonstrate in any systematic way how, in health promotion, various methods are employed to reach an entire population or major target groups. Other authors (Cohen, 1970; Haglund, 1990) further lament over the low status of process evaluations, and thereby the difficulties involved in financing and developing qualitatively good research.
In their meta-study, however, Nutbeam et al. identify three areas within the implementation (or process) of public-health work that prove to be describable and/or measurable (Nutbeam et al., 1990). These areas can be summarized under the headings of network, exposure and commitment. Bjaras and Haglund have developed these lines of thought, and show moreover how process participation can be described (Bjaras and Haglund, 1990).
Network
A network is utilized to spread the message within a community or to a target group. The importance of building a socially effective network has been confirmed in communications research (Bandura, 1977).
Research into general medical practitioners' and district nurses' roles as disseminators of information (Rosen, 1992) has, for example, demonstrated both obstacles to and opportunities for the implementation of previously tested methods. Deliberate networking and information-dissemination activities are essential to most programmes and projects. The literature demonstrates the importance of skills development within the network (Haglund and Svanstrom, 1992). Windahl et al. point to the importance of nurturing passionately committed local enthusiasts (Windahl et al., 1991). If such people do not obtain due recognition for their efforts, their passion can easily abate or, even worse, they can be transformed into counteracting forces. One way to investigate quality is to find out whether and how a programme includes networking in its operations.
Exposure
Programme exposure is concerned with the extent to which and with what intensity target groups come into contact with health information and health-promoting activities, and to what degree target groups receive the message. The difficulty in measuring programme exposure increases with the degree of community intervention. Experiments that have been conducted in Pawtucket, Massachusetts (Lefebvre, 1990), among other places, show however that it is possible to describe exposure in many ways.
Commitment
Commitment is a summary concept designed to embrace attempts to measure how well rooted and accepted the health message is among those who participate in the programme. If networks and messengers are to deliver their message they must be able to embrace it themselves.
Several studies have shown that it is possible to measure message acceptance. Similarly, there is an opportunity to measure how well a programme's ideas are rooted among its collaborating partners and those who relay them further within the community. Research in this area has been conducted primarily among teachers and so-called ‘peer educators’ (Marklund, 1994).
Under this heading, the appearance of what might be called ‘counteracting-forces’ should be considered (Pincus and Minahan, 1973). These can be identified through insufficient level of support, but also through more active opposition, and via structural problems.
Participation
Participation means that the partners affected, key members of the target group, decision-makers and other persons affected have all been equipped with an opportunity to influence and take part in the programme.
Social efficacy is a key concept (Bandura, 1977) in the field of health communication. For it to be achieved, groups' and persons' social skills must be made use of and influence programme design. Bjaras and Haglund show how participation can be described and measured within different parts of a programme (Bjaras and Haglund, 1990). Arnstein's ‘ladder of participation’ also provides an analytical instrument for following up quality by means of concepts such as symbolic and actual participation (Arnstein, 1971).
Outcome
It should be possible to refer the outcome of a primary-preventive programme not only to the goals that have been set up for a programme but also to the efforts that have been made, i.e. to both the programme's structure and its process. If the quality of work conducted within the programme is good, this should raise the quality of its results.
The outcome of a population-oriented preventive programme should be measurable in changes in epidemiological variables, i.e. changes in morbidity, mortality, or, for example, the incidence of injury events. Often, however, this is not possible, e.g. when other societal changes are working in the opposite direction or because health promotion often only achieves tangible results after a long period of time. In this case, measurements of behavioural changes in target groups and/or environmental or societal changes can be employed as outcome indicators. In many cases, changes in various risk factors, such as smoking habits, alcohol consumption, body weight/ body mass index or high blood pressure, are more appropriate as outcome measures than solely the incidence of disease (Rosén, 1992).
Effects on and changes in health in a population are frequently the result of the impacts of a variety of factors. For this reason, it is important, if possible, to take measurements both before and after the implementation of the programme/the intervention. An analysis should also be made of whether the changes are due to the intervention or to other factors (Haglund, 1990). Maintenance of the programme/the project itself, just as the results achieved, also form a part of the quality follow-up (Lefebvre, 1990). The indicators of a programme's outcome are therefore as follows.
Knowledge and behavioural changes
Behavioural changes measure change in the target group's knowledge, attitudes and behaviour.
Environmental changes
Environmental changes show changes in the surrounding community environment, such as adaptations to children's playgrounds or the introduction of separated thoroughfares for pedestrians, cyclists and motor vehicles.
Epidemiological changes
Epidemiological changes show changes in the consumption of care, morbidity or mortality.
Maintenance
Model projects in the arena of health promotion have proved to have a limited long-term effect if project work is not incorporated into regular organized activities following project completion.
TESTING THE FUNCTIONALITY OF THE INDICATORS AND THE QUESTION PRO-FORMA
A test of the question pro-forma was done on a well established injury prevention programme, awarded with the Safe Community Award by WHO (Ader et al., 1992b). The result of this test is summarized below. There follows an extensive account of quality assurance with regard to the injury prevention programme. This is designed to show the types of arguments to which the questions give rise. The test was conducted by means of a group of persons with project-management responsibilities answering questions on the pro-forma.
Test of indicators of the programme's structure
Goals
In general, the programme had distinguishable goals, but these were not structured or specified, e.g. with regard to time limits for implementation or measurability. The review had the result that the programme's goals were reformulated and given a greater degree of precision.
Target groups
The programme had selected target groups, which were clearly defined. The selection had been made on the basis of injury registration that had taken place both regionally and nationally. This registration indicated that the selected target groups had been exposed to high risks and did not possess any organization to protect their interests in this respect.
Design
A model, which had been elaborated in the pioneering Falkoping experiment in Sweden (Schelp, 1987) provided the foundation for the programme in Lidkoping. A local model for practical activities also existed. Quality assurance has been built into the programme since review. A special quality control has been performed with regard to injury registration while, at the same time, the programme was scrutinized from an ethical perspective.
Responsibility
Division of responsibility was clear at management level, but there was a certain lack of clarity at operative level. Following the review, responsibility for various interventions, measures and follow-up has been clarified.
Resources
There were no special funds ear-marked for the programme and needs were covered within the frame of a larger general allocation for activities in the health promotion arena. No analysis of skills needs on the part of personnel had been conducted. Skills-development resources and supervision were available, as had been the case throughout the programme. But resources were lacking for analysis and development of the epidemiological surveillance system. All these problems were made evident by the quality assurance.
Organization
Organization and mandate had been clear from the outset within the health authority, but not within the local authority. High ambitions had sometimes led to delays. As a result, more detailed scheduling was introduced, and there is now a specification of which measures are to be regarded as parts of continuous operations.
Test of indicators of the programme's process
Network
The network principally consisted of parts of the health and local authorities. Voluntary organizations were linked to the programme through the local authority's contacts. There were no opportunities for the network to evaluate its own work. There was no systematic identification of local enthusiasts. Some leading community ‘spirits’ were encouraged, in among other ways, by being given access to participation in conferences and seminars. The need for development of different skills in the network became obvious through the review.
Commitment
Persons participating in the programme entered into an informal contract in which they took on responsibility for the implementation of either the programme as a whole or part of it.
No measurements of how the network accepted the message had been taken; nor had any assessment been made of the level of acceptance. Feedback of information took place through annual management reports. No structured activity was undertaken for the identification of possible counter-forces. The need for work in the area of commitment was highlighted by the quality assurance activity.
Exposure
An example of activities and recording is an interview survey conducted in 1989, covering a total of 480 community residents. The local authority office had a hot-line for tips from the public on how injury risk could be reduced. Child-health clinics offered a so-called ‘bicycle-helmet prescription’ to children. Home-help personnel had a special form for observations and actions taken in the residences of the elderly, and the information was then passed on to supervisors.
On the part of the network there were certain opportunities to measure how the message had come across, but not how the network itself perceives the message.
Participation
It was possible to measure participation on the basis of memoranda and management reports, but no formal follow-up was conducted. In one of the intersectoral groups, attendance of group meetings was mandatory. Otherwise, no demands were made for active participation.
Test of indicators of the programme's outcome
Behavioural changes
The number of bicycle helmets purchased for children was reported each year. A questionnaire had been administered to parents, containing a question on whether their children used helmets when bicycling. Otherwise, no measurements were performed.
Environmental changes
The review demonstrated that there was no comprehensive instrument for demonstrating environmental changes in the community.
Epidemiological changes
The injury registration that had taken place at the hospital, in health-care centres and at emergency medical units since 1989 had enabled the trend in the number of accidents requiring medical care to be monitored.
Maintenance
Work was co-ordinated by the intersectoral group. Parts of the action programme had been incorporated into the regular operations of the municipality. There were routines for the continuous reporting, inspection and maintenance of playgrounds so that injury hazards could be removed; also, child minders were provided with check lists for the prevention of accidents in the home. Municipal politicians also reached a decision that full-scale model residences for the elderly should be constructed and tested in advance of use as a means of providing a safer environment for elderly people.
FURTHER EXPERIENCES OF USING THE QUESTION PRO-FORMA
The question pro-forma has been simplified into an easy-to-use template, which has allowed more extensive use in Sweden (Berensson et al., 1994). Several follow ups of how the model can be applied have been conducted, as well as a follow-up of how it has been used in Sweden.
One test showed that the model could be successfully applied into different fields of health promotion: a project for elderly, an alcohol and drug project and a weight control project (Stockholms läns landsting, 1998).
A study focused on areas of application and on functionality was done on 60 users of the question pro-forma by the Swedish National Board of Health and Welfare, proving, among other things, the usefulness of the method for intersectorial collaboration (SNBHW, 1999). This study shows among other things that the template is the most used method for quality assurance within the health promotion area in Sweden. According to the study, all users found that the quality of their projects had benefited from the template. It especially provided good support for intersectorial approaches.
The conclusion of the study is that the template is a useful tool for systematic work within health promotion, and therefore enables planning and evaluation.
DISCUSSION
By seeking common characteristics among successful health promotion programmes/projects we have found 14 general indicators that are applicable in quality assurance. Operationalizing the quality indicators has contributed to the creation of a question pro-forma specifically designed to facilitate quality assurance work. By testing the quality indicators and question pro-forma on different programmes, of which one was presented above, we have demonstrated that the method is utilizable.
We take the view that the proposed indicators and question pro-forma are so designed that they can have general application in the quality assurance of health promotion programmes and projects. The quality indicators provide an opportunity, during the period a programme is running, both to test the programme as it is, and where there are grounds for this, to effect appropriate modifications to its design or procedures.
The reported tests of the indicators and the question pro-forma and a template have provided a basis on which persons in charge of a programme can change programme arrangements for the better, and changes have actually taken place. The question pro-forma can be used with varying degrees of ambition. A possibility for the future could be to adapt it to more specific areas of health promotion.
A development for the future could also be to construct more specific questions regarding client participation. Although the template is easy to use, there is a need for training as well as time and resources for practitioners to implement it within their daily work.
Since the question pro-forma was developed, many authors have discussed the possibilities and constraints of quality assurance in health promotion (David and McDonald, 1998) that may require some specific operationalization for a particular programme. In several countries, efforts have been undertaken to develop methods for this purpose (EU Country Reports, 1996; Speller et al., 1997). Many of these attempts point in the same direction as the result of the study in this article.
The indicators and the question pro-forma can be regarded as manifestations of ‘sensitizing concepts’ (Cohen, 1970), i.e. general concepts that may require specific operationalization for each particular programme. They may also need to be tailored to the various phases or stages of a programme or project.
Note
The question pro-forma is available in different documents:
- Folkhalsovetenskap—en introduktion (Science of Public Health—an Introduction), 1992 (Haglund and Svanstrom, 1992);
- Kvalitetssakring av primarpreventiva program på befolkningsniva (Quality Assurance of Health Promotion Programs), 1992 (Ader et al., 1992a; report from the Public Health Unit in County Council Skaraborg);
- ‘Att lyckas med folkhalsoprojekt’ (Berensson et al., 1994), which has been translated into English, Succeeding with Health Promotion—Quality Assurance (Berensson et al., 1996);
- Succeeding with Health Promotion is also available on the Internet at http://www.lf.se, in both Swedish and English.
| An example of the question pro-forma to indicate target group is as follows.
|
| An example of the question pro-forma to indicate participation is as follows.
|
An example of the question pro-forma to indicate changes in knowledge and behaviour is:
An example of the question pro-forma to indicate environmental changes is as follows. Is there a systematic recording of:
|
REFERENCES
Ader, M., Beresson, K., Carlsson, P., Enhorn, G., Granath, M. and Urwitz,V. (1992a) Kvalitetssäkring av primärpreventiva program på befolkningsnivå. Report from Skaraborg County Council, Skövde 1992.
Ader, M., Ekman, R., Hagaeus, A.-C. and Sjogren, D.-E. (1992b) En säker och trygg kommun, Lidkoping (Safe Community), Lokalt olycksfallsprogram 1991, Lidkopings kommun och Landstinget Skaraborg, Lidköping (stencil).
Altman, D. (1986) A framework for evaluating community-based heart disease prevention programs. Social Science and Medicine, 22, 479–487.
Arnstein, S. (1971) Eight rungs on the ladder of participation. In Cahn, P. (ed.) Citizen Participation: Effecting, Community, Change. Praeger, London.
Bandura, A. (1977) Social Learning Theory. Prentice Hall, Engelwood Cliffs, NJ.
Berensson, K., Granath, M. and Urwitz, V. (1994) Att lyckas med hälsoprojekt—Kvalitetssäkring, Landstingsförbundet, Stockholm.
Berensson, K., Granath, M. and Urwitz, V. (1996) Succeeding with Health Promotion Projects—Quality Assurance, Landstingsförbundet, Stockholm.
Berwick, D. M., Godfrey, A. B. and Roessner, J. (eds) (1990) Curing Health Care: New Strategies For Quality Improvement. First edition. Jossey-Bass Publishers, San Francisco, CA, pp. 46–66.
Bjärås, G. and Haglund, B. J. A. (1990) Att mäta medborgardeldagandet i ett samhällsinriktat hälsoprogram. Socialmedicinsk Tidskrift, 5–6, 266–271.
Bracht, N. (ed.) (1990) Health Promotion at the Community Level. Sage Publications, Inc., Newbury Park, CA.
Calltorp, J. and Bergström, M. (1992) Medicinsk revision, Stockholms läns landsting, Stockholm.
Cohen. (1970) What is wrong with social theory? In Filstead, W. J. (ed.) Qualitative Methodology. Markham, Chicago, IL.
Cullen and Hollingum (1987) Implementing Total Quality. IFS Publications Ltd, Springer-Verlag, Köln.
David, J. K. and McDonald, G. (1998) Quality Evidence and Effectiveness in Health Promotion. Routledge, London.
Donabedian, A. (1996) Evaluating the quality of medical care. Milbank Memorial Fund Quarterly, 44, 166–206.
EU Country Reports (1996) Overview of quality assurance and health promotion interventions in the EU-member States in 1996. IUHPE/EURO.
Fisker, J. (1992) Experiment as a Strategy for Change. AKF memo. KKF Förlaget, Copenhagen.
Fry, D. (1990) Systems of standards for community health services in Australia. Quality Assurance in Health Care, 2, 59–67.
Green, L. W. and Lewis, F. M. (1986) Measurement and Evaluation in Health Education and Health Promotion. Mayfield Publishing Company, Palo Alto, CA.
Haglund, B. J. A. (1990) Folkhälsoarbetets framtida forskningsfrågor. Socialmedicinsk Tidskrift, 5–6, 301–310.
Haglund, B. J. A. and Svanström, L. (1992) Folkhälsovetenskap—en introdukton, Studentlitteratur, Lund.
Johnston, M. (1980) Appendix 1: development of a community health program. CONTACT, special series No. 3. World Community of Churches, Geneva.
Key, M., Hudson, P. and Armstrong, J. (1976) Evaluation Theory and Community Work. Community Projects Foundation, London.
Landstingsförbundet (1994) Kvalitet i vården. Instrument för systematisk utveckling och utvärdering. Landstingsförbundet, Socialtyrelsen, SPRI, Arbetslivsfonden, Stockholm, Sweden.
Lefebvre, R. C. (1990) Strategies to maintain and institutionalize successful programs, a marketing framework. In Bracht, N. (ed.) Health Promotion at the Community Level. Sage Publications, Inc., Newbury Park, CA.
Macdonald, G. (ed.) (1992) Quality in Health Promotion. Report of the Annual Partnership Conference, Buckland Hall Powys. Health Promotion Authority for Wales, Cardiff.
Marklund, U. (1994) Peer-Education, HIV-Aktuellt. Folkhälsoinstitutet, Stockholm.
Neuhauser, D., McEachern, J. and Headrick, L. (1995) Clinical CQI. A Book of Readings. Joint Comission on Accreditation of Healthcare Organizations, Oakbrook Terrace, IL.
Nutbeam, D., Smith, C. and Catford, J. (1990) Evaluation in health education, a review of progress, possibilities and problems. Health Promotion Authority for Wales, Cardiff.
Pincus, A. and Minahan, A. (1973) Social Work Practice: Model and Method. Peacock Publishers, Inc., Itasca, IL.
Puska, P., Nissinen, A. and Tuomilehto, J. (1985) The community-based strategy to prevent coronary heart disease: conclusions from the 10 years of North Karelia Project. Annual Review of Public Health, 6, 147–191.[Web of Science][Medline]
Rosén, M. (1992) Kvalitetssäkring av hälsofrämjande och förebyggande arbete, PM. Institutionen för socialmedicin, Karolinska Institutet, Stockholm (stencil).
Sanderson, C. and Svanström, L. (1985) The Stockholm Cancer prevention program. Framework for evaluation, Grey report 118, Stockholm County Council Public Health Services, Stockholm.
Schalma, H. and Kok, G. (1995) Promoting Health through Education. Odyssey volume 1, issue 2, University of Nijmegen.
Schelp, L. (1987) Epidemiology as a Basis for Evaluation of a Community Intervention Program on Accidents. Thesis, Institutionen för socialmedicin, Karolinska Institutet, Stockholm.
Speller, V., Evans, D. and Head, M. (1997) Developing quality assurance standards for health promotion practice in the UK. Health Promotion International, 12, 215–224.
Stockholms läns landsting (1998) Att kvalitetsutveckla folkhälsoarbetet.
Windahl, S. (1991) Når budskapet ut? Utvärdering av LAFA, KAN, kommunikationsanalys, Stockholm (mimeo).
Swedish National Bord of Health and Welfare (1999) Uppföljning av kvalitetsmanualen ‘Att lyckas med folkhälsoprojekt’.
World Health Organization (1983) The principles of quality assurance. WHO, Geneva.
CiteULike 
Connotea 
Del.icio.us What's this?
This article has been cited by other articles:
|
|
 |
|
  C. K. van der Ent Quality assessment: is the truth in the outcome? Eur. Respir. J., January 1, 2008; 31(1): 6 - 7. [Full Text] [PDF]
|
|
|
|
 |
|
 P Nilsen The theory of community based health and safety programs: a critical examination. Inj. Prev., June 1, 2006; 12(3): 140 - 145. [Abstract] [Full Text] [PDF]
|
|
|
|
 |
|
 A. A. Aro, S. Van den Broucke, and S. Raty Toward European consensus tools for reviewing the evidence and enhancing the quality of health promotion practice Global Health Promotion, March 1, 2005; 12(1_suppl): 10 - 14. [PDF]
|
|
| ||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||