Health Promotion International Advance Access originally published online on December 17, 2007
Health Promotion International 2008 23(1):24-34; doi:10.1093/heapro/dam036
| ||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||
People and money matter: investment lessons from the Ontario heart health program, Canada
1Centre for Behavioural Research and Program Evaluation, Lyle S. Hallman Institute N 2Department of Health Studies and Gerontology, University of Waterloo, Waterloo, Ontario, Canada N2L 3G1 3School of Nursing and Community Health Research Unit, University of Ottawa, Ottawa, Ontario, Canada 4 Health Promotion Consulting Inc., Kitchener, Ontario, Canada
* Corresponding author. E-mail: briley{at}healthy.uwaterloo.ca
 |
SUMMARY |
|---|
 TOP SUMMARY INTRODUCTIONMETHODSRESULTSDISCUSSIONCONCLUSIONFUNDINGREFERENCES |
|---|
Resource allocation is a critical issue for public health decision-makers. Yet little is known about the level and type of resources needed to build capacity to plan and implement comprehensive programs. This paper examines the relationships between investments and changes in organizational capacity and program implementation in the first phase (1998–2003) of the Ontario Heart Health Program (OHHP)—a province-wide, comprehensive public health program that involved 40 community partnerships. The study represents a subset of findings from a provincial evaluation. Investments, organizational capacity of public health units and implementation of heart health activities were measured longitudinally. Investment information was gathered annually from the provincial government, local public health units and community partners using standard reports, and was available from 1998 to 2002. Organizational capacity and program implementation were measured using a written survey, completed by all health units at five measurement times from 1994 to 2002. Combining provincial and local sources, the average total investment by year five was $1.66 per capita. Organizational capacity of public health units and implementation of heart health activities increased both before and during the first 2 years of the OHHP, and then plateaued at a modest level for capacity and a low level for implementation after that. Amount of funding was positively associated with organizational capacity, yet this association was overpowered by the negative influence of turnover of a key staff position. Regression analysis indicated that staff turnover explained 23% of local variability in organizational capacity. Findings reinforce the need for adequate investment and retention of key staff positions in complex partnership programs. Better accounting of public health investments, including monetary and in-kind investments, is needed to inform decisions about the amount and duration of public health investments that will lead to effective program implementation.
Key words: resource allocation; health promotion; primary prevention; health resources
|
INTRODUCTION |
|---|
|
TOPSUMMARYINTRODUCTIONMETHODSRESULTSDISCUSSIONCONCLUSIONFUNDINGREFERENCES |
|---|
Strengthening public health and health promotion infrastructure is a priority in Canada and internationally [cf. (Health Canada, 2003
; Mittelmark et al., 2005)]. Infrastructure includes many structures and processes. For example, Canada's public health renewal plan includes four core elements: organizational capacity; public health workforce; business processes, information and knowledge systems; and strategic capacity (Health Canada, 2003). Within and beyond Canada, funding is recognized as a necessary resource to support public health renewal. However, funding for public health is described as modest (at best) by any measure, and reliable information on public health expenditures is not readily available (Health Canada, 2003). Thus, evidence is limited on levels of investment required to achieve public health program objectives. This is a problem for public health decision-makers who are called on to justify budgetary allocations for public health services (Mays et al., 2004). A key area of public health services lacking information about optimal levels of investment is chronic disease prevention (Mirolla, 2004).
Specific to chronic disease prevention, some investment guidelines are available in the area of tobacco control. For example, for a population of 10 million, recommended annual per capita investments in 1999 were $5–16 US (Centres for Disease Control, 1999) and $8 Canadian (Ashley et al., 1999). These investment recommendations were for a comprehensive set of programs and services, including elements such as capacity building (i.e. technical assistance and training), local programs, media and policy development. Relevance of these estimates for multi-risk factor programs, such as those programs targeting heart health, is largely unknown.
Studies informing the tobacco investment guidelines examined comprehensive, state-wide tobacco control programs that assessed reductions in smoking as evidence of effectiveness. Dissemination research has identified several important, more proximal measures that contribute to risk factor changes. Dominant among these are measures of organizational capacity and program implementation (NSW Health, 2001; Lafond et al., 2002; Germann and Wilson, 2004; Joffres et al., 2004).
Organizational capacity to undertake community-based programs has a direct and strong influence on program implementation (Riley et al., 2001). Consistent with a social ecological perspective (Green et al., 1996), other factors internal to organizations (e.g. priorities, planning structures, staff turnover) and within the external environment (e.g. social and physical characteristics of communities, political priorities) also influence organizational performance (Champagne et al., 1993; Riley et al., 2001; Joffres et al., 2004; MacVicar, 2006). Two main challenges include loss of personnel in key positions [cf. (Ulrich et al., 2005)] and remote geographic locations (Robinson and Elliott, 1999).
The purpose of this paper is to contribute to the knowledge base on the investment in chronic disease prevention using heart health as a case example. Heart health is an instructive case since it addresses common risk factors for several chronic diseases and it applies ‘best practices’ in public health, including capacity building and implementation of comprehensive community programs (Centres for Disease Control, 1999). In this paper, relationships are examined among investments, organizational capacity of Ontario's public health units and community-wide implementation of heart health activities. The impact of staff turnover and geographic location are also examined. The study extends dissemination research of the Canadian Heart Health Initiative (Canadian Heart Health Dissemination Research, 2001), and represents a subset of findings from a provincial evaluation of the first phase of the OHHP, which was implemented from 1998 to 2003.
Research setting
Ontario is Canada's largest province with approximately 13 million people. Public health services are primarily delivered through regional health units, each administered by a board of health, and regulated by provincial legislation, including mandatory core program guidelines. Most public health programs are cost-shared by provincial and municipal governments, with most resources allocated to personnel. The public health funding envelope is 
2% of the total provincial health care budget (Health Canada, 2003). Within the public health budget, a relatively small proportion is allocated to promoting tobacco use reduction, physical activity and healthy eating. Prior to the start of the OHHP, no more than 10% of public health resources were allocated to these activities (Schabas, 1996). In the late 1990s, the Ministry of Health and Long-term Care (MoHLTC) supplemented the base funding for selective public health programs. The OHHP was one of these programs.
The Ontario Heart Health Program (1998–2003)
The OHHP aimed to reduce leading modifiable risk factors for cardiovascular and other chronic diseases. The Program built on a provincial demonstration phase (Heart Health Action Program, 1990–1996) and a dissemination research phase (Canadian Heart Health Initiative Ontario Project, 1994–1998). It also met some requirements of mandatory public health programs in chronic disease prevention. Local OHHP programs were delivered by community coalitions (referred to as community partnerships), mostly consisting of agency representatives from health and non-health sectors. The geographic boundaries of the OHHP community partnerships matched those of the public health units. Activities of partnerships were guided by and conducted within a common technical and administrative provincial framework, including a dedicated resource centre (for technical assistance and training), provincial and regional heart health networks, program coordination at the MoHLTC, and an evaluation team. The OHHP was cost-shared between the MoHLTC, local boards of health and other community agencies. A funding requirement was that each provincial dollar be matched with two local dollars. Part of the local contribution required for each partnership was funding for a full-time Coordinator.
Implementation context
Many initiatives complemented the OHHP. Dominant among these were a public health mandate in chronic disease prevention that was first introduced in 1989, 100% provincial funding for tobacco programming as part of the Ontario Tobacco Strategy and a provincial resource system (consisting of several resource centres including the one dedicated to the OHHP) to strengthen local health promotion capacity. However, the Ontario public health system was also considerably strained and in flux during this same time period. The system was under-resourced. Provincial strategic planning for chronic disease prevention was limited. Provincial government branches for public health and health promotion were merged. In 1997, the provincial-municipal cost shared formula for public health funding was changed to 100% funding for public health programming by municipalities (Note: since the OHHP, the cost-shared arrangement for public health services in Ontario has been re-instated). Some health units were amalgamated. Also during this time, several unanticipated and major public health crises occurred in Ontario, including contamination of the Walkerton water supply system (May 2000), West Nile virus (2001–2002) and SARS (2003).
|
METHODS |
|---|
|
TOPSUMMARYINTRODUCTIONMETHODSRESULTSDISCUSSIONCONCLUSIONFUNDINGREFERENCES |
|---|
Hypotheses
This study examines provincial-level trends in investment, organizational capacity and program implementation before and during the 5-year time period of the OHHP. Positive trends were anticipated between investment, organizational capacity and program implementation, with a time lag from when OHHP investment occurred to when organizational capacity increased, and a subsequent time lag from when capacity increased to when program implementation increased. Negative associations were anticipated between staff turnover and northern location on one hand, and organizational capacity on the other.
Study design, variables and data sources
The OHHP evaluation used a multi-method longitudinal design. Table 1 shows the measurement times. Geographic location (northern, non-northern) was based on regional classifications of public health jurisdictions in Ontario. Investments and staff turnover were estimated annually during the OHHP. Capacity of public health units and community implementation of heart health activities were measured three times in the 4-year period prior to the OHHP and two times during OHHP implementation (Table 1). Findings reported in this paper are a subset from the OHHP evaluation. Relevant procedures received ethics approval from the University of Waterloo Human Research Ethics Committee. Interpretation of findings was done in consultation with public health research, policy and practice stakeholders working within the Ontario context as a way to maximize social validity (Lomas, 2000).
|
Investment
Table 2 shows data collection procedures for OHHP provincial and local investments. Investment data about provincial program components (e.g. resource centre, administration, evaluation) were provided by OHHP staff in the MoHLTC. Investment data about local program components were provided by community partnerships. Standard local reports were used, which documented expenses from the provincial funding and in-kind contributions from local boards of health and other agencies. Local reports were completed by the OHHP Coordinator using standard definitions and guidelines, and approved by the Chair of the community partnership. The only variability in data collection was how each community partnership gathered information on in-kind contributions from partner agencies (e.g. estimated from OHHP Coordinator, written reports from each participating agency). Total annual investments in local programs represent the sum of expenses from the provincial grant, and an estimated dollar value of reported in-kind contributions.
|
Staff turnover
In the annual reports, the start and end dates for each OHHP Coordinator were recorded. From these data, total number of Coordinators in each health unit over the 5 years was determined.
Organizational capacity and program implementation
Province-wide surveys of public health departments measured organizational capacity and implementation of heart health activities at five measurement times between 1994 and 2002. Organizational capacity refers to skills and resources required to implement community-based heart health activities. Implementation refers to the performance of community-based heart health activities in defined geographic areas (i.e. not limited to those activities undertaken as part of the OHHP). Satisfactory reliability (i.e. internal consistency) and construct validity were established for both organizational capacity and implementation of heart health activities [for more details, see (Riley et al., 2001)]. Tables 3 and 4 provide examples of items and the rating scales for these measures.
|
|
The organizational response was determined jointly by the local Medical Officer of Health and the staff who were most involved in heart health promotion. The response rate for this survey was 100% at all five data collection times. Survey procedures were consistent with guidelines for collecting organizational level data (Steckler et al., 1997
) and followed the same procedures over time with one exception. In 1994, two surveys (a corporate survey and an individual survey) were administered, whereas in subsequent years, these surveys were combined into a single corporate survey. For all years, corporate scores were used in analyses. For 1994, corporate scores for organizational capacity were derived by averaging scores from individuals within units after confirming strong correlations between individual scores and within-unit means (Shrout and Fleiss, 1979; James, 1982). Data for surveys administered from 1994 to 1997 were used with permission from the investigators of the Canadian Heart Health Initiative Ontario Project.
Regression analysis
A correlation matrix of all variables was assembled. The dependent variable for multiple regression analysis was organizational capacity in 2002. The explanatory variables included per capita funding from all sources (including the MoHLTC, local boards of health and community partnerships) in 2001/2002; number of OHHP Coordinators over all 5 years; and geographic location (northern, non-northern). A forward stepwise procedure was used.
|
RESULTS |
|---|
|
TOPSUMMARYINTRODUCTIONMETHODSRESULTSDISCUSSIONCONCLUSIONFUNDINGREFERENCES |
|---|
Investments in the Ontario Heart Health Program
Combining local and provincial OHHP investments, average annual investments in the OHHP increased each year, starting at approximately $1.20 per capita (range from $0.35 to $6.85) in 1998 and reaching $1.60 per capita ($0.49 to $6.22) in 2003 (Figure 1). These increases were from local in-kind contributions. Provincial investments were the same in each health unit each year.
|
Organizational capacity for heart health promotion
Organizational capacity increased steadily before the OHHP, and then plateaued during the OHHP (Table 5, Figure 2). By the end of the OHHP, health unit ratings of organizational practices ranged from 1.4 to 3.8 corresponding to just over ‘not very effective’ to ‘very effective’ descriptors on the scale. The average rating corresponded to ‘somewhat effective’.
|
|
Implementation of heart health promotion activities
Province-wide implementation of heart health activities increased slowly but steadily from 1994 to 2000, and then plateaued from 2000 to 2002 (Table 5, Figure 2). By the end of the OHHP, health unit ratings of implementation ranged from 1.4 to 2.9, corresponding to more than ‘active planning’ but less than a ‘medium level’ of implementation. The average community was at a ‘low level’ of implementation of heart health activities.
Predictors of organizational capacity
All variables were significantly correlated with organizational capacity. However, in the final regression model, only number of Coordinators was retained (β = –0.13; t[39] = –2.14; P = 0.04; R2 = .23). Thus, staff turnover explained 23% of the local variance in organizational capacity.
|
DISCUSSION |
|---|
|
TOPSUMMARYINTRODUCTIONMETHODSRESULTSDISCUSSIONCONCLUSIONFUNDINGREFERENCES |
|---|
Amount of investment
The expected trends of increased organizational capacity and increased level of implementation corresponding to increased investment (during the five years of the OHHP) were not observed. Capacity and implementation scores increased steadily before the OHHP investment and tapered off during the 5-year OHHP. Interpretation of these trends must consider the many program and contextual factors that likely influenced the ‘success’ of comprehensive public health programs (e.g. intervention fidelity, intensity of intervention, reach, timing in relation to policy windows and political priorities) (Joffres et al., 2001; Edwards, 2006
).
Gains before 1998 (year 1 of the OHHP) were likely due to a major policy change in public health, and other health promotion developments in Ontario. In 1989, healthy lifestyles programs (including tobacco use prevention, nutrition promotion, physical activity promotion) were introduced into the public health mandate. These programs charted a new course in public health towards population-level interventions (such as environmental and policy change). At the same time, a health promotion resource system was developing in Ontario with a mandate to increase local capacity for health promotion. The Ontario Tobacco Strategy was launched in 1992 (including provincial funding for local programs). In 1993, the report of the Chief Medical Officer of Health for the province was on promoting heart health (Ontario Ministry of Health, 1993). A federal-provincial heart health initiative was conducted in 1994–1998, focusing on strengthening local capacity for community interventions. By 1996–1997, health units were planning for the OHHP (Riley, 2003). In this increasingly complex health promotion environment in Ontario, the measures of capacity and implementation may not have been sensitive enough to pick up more subtle changes that occurred during the OHHP (Riley et al., 2004).
In addition, Ontario stakeholders reinforced the importance of interpreting the observed trends in organizational capacity and implementation in the broader context of public health and socio-political and economic conditions prior to and during the OHHP implementation period. With respect to capacity, the most plausible interpretation shared by stakeholders was that the OHHP prevented backsliding during the study period. The 5 years of the OHHP were peppered with challenging circumstances for public health in general and chronic disease prevention in particular. In addition to low levels of resources, turnover of OHHP Coordinators was high, the organization of public health at the provincial level was restructured, and many competing priorities were apparent. Consistent with results from acute care settings (Irvine and Evans, 1995; Aiken, 2001; Strachota et al., 2003; Ulrich et al., 2005), staff turnover had a particularly potent influence on organizational capacity, overpowering associations with investment and geographic location.
Other explanations for the plateau in organizational capacity during the OHHP could be that public health capacity peaked before the OHHP, or that a threshold was reached such that no further gains in capacity were needed to increase program implementation. These explanations, however, were not convincing. Stakeholders agreed that further growth in organizational capacity was possible and needed (Riley et al., 2004).
With respect to levels of implementation, a similar explanation is plausible—for the pre-OHHP gains and the plateau from 2000 to 2002. As expected, levels of implementation were lower than levels of organizational capacity, which supports the time lag hypothesis. However, the gap may be smaller than the scores imply since, arguably, ‘ideal implementation’ is unattainable due to limited resources and other constraints. Also, public health staff may have lowered their implementation ratings as the project progressed, because they more fully realized the potential scope of activities that could be implemented due to increased knowledge or discussions with colleagues.
Few comparisons are available from other studies to understand the associations between the amount of investment on one hand, and capacity and implementation on the other. The recommended investments in the area of tobacco control provide some basis for comparison, even though a comprehensive approach for a single risk factor is an imperfect comparison to a comprehensive approach for multiple risk factors. Assuming spending recommendations for multiple risk factors would not be less than those for tobacco, the investment in the OHHP does not fare well. A liberal estimate for per capita expenditures on heart health in Ontario would include major local and provincial investments in heart health and tobacco control. In the final year of the OHHP, if investments in the OHHP are combined with the Ontario Tobacco Strategy (estimated at $1.41 per capita in 2002–2003) (Ontario Tobacco Research Unit, 2005), and resource centres directly relevant to heart health promotion (i.e. program support for tobacco, nutrition and physical activity) (estimated at 10 cents per capita, derived from an estimated investment of $1 million in resource centers for a population of 12 million, which is extrapolated from funding for the Heart Health Resource Centre). Ontario investment in heart health (including funding for programs and staff) was less than $4 per capita in 2003. (Note: this estimate does not include federal investments in tobacco control or in-kind contributions, which are assumed to be very low on a per capita basis.)
Sustainability of investment
There are three considerations in looking at sustainable investments in heart health and the impact on programming. The first is the duration of investment. The second is the certainty of funding during and beyond the initial investment period. The third is the retention of key staff positions. The low to modest absolute levels of organizational capacity and program implementation (respectively) suggest that one or more aspects of sustainable investment were insufficient to observe changes in the desired outcomes.
Regarding the first and second consideration, a 5-year funding allocation covers a political cycle but is a short period of time for community interventions (Merzel and D'Afflitti, 2003). In addition, the OHHP funding was renewed annually. Experience suggests that time-limited funding, or short-term renewable funding arrangements create cycles of growth and destruction within partnerships, and will not lead to effective and sustained local programming (Orlandi, 1996; Dunkley et al, 2001). Worse, if repeated, these cycles can lead to frustration and cynicism, and interfere with future efforts to work together.
With respect to the third consideration, perhaps the most striking and convincing finding, is the negative influence of staff turnover on organizational capacity. This finding is supported by research undertaken in acute care hospitals in which staff turnover was shown to have high economic costs (Jones, 2005) and to reduce capacity to provide quality patient care (Shields and Ward, 2001; Larrabbee et al., 2003). The results in this paper begin to build the evidence base on the impact of staff turnover in public health. Examining the reasons for the turnover in the OHHP and other similar initiatives are needed to inform the recruitment and retention of appropriate staff [cf. (MacVicar, 2006)], and the direction of the relationship between staff turnover and organizational capacity. For example, do staff leave due to frustrations related to low organizational capacity, or does staff turnover negatively effect organizational capacity, or both.
Study strengths and limitations
The results presented in this paper build on previous work by providing longitudinal data on levels of investment, organizational capacity and program implementation. They also incorporated indicators of organizational capacity and program implementation with established reliability and validity, and collected comprehensive cost data from multiple sources. Nevertheless, the OHHP evaluation represents data from one province and for a relatively short period of time in relation to the long-term change process required for comprehensive public health programs (Greenhalgh et al., 2004; Miller and Shinn, 2005). Also, although investment data were suitably collected from multiple sources, accuracy of local in-kind contributions may have been compromised by two factors. One, there may have been an over-reporting bias since a requirement of the OHHP was for local agencies to contribute the equivalent of two local dollars for every one provincial dollar. Variability in procedures to collect local in-kind data also may have decreased accuracy.
Another limitation is the ‘black box’ between investments and changes in organizational capacity and program implementation; that is, an assessment of how the investments were used. The OHHP evaluation included a detailed tracking of program implementation and some tracking of contextual influences. Although the findings are not reported in this paper, they informed the interpretation of trends over time (Riley et al., 2004). The analysis team was satisfied that OHHP investments were indeed used as intended (for technical assistance and training and for local programs), and directed to the intended program activity areas. This condition needed to be met to ensure a meaningful association between OHHP investments and indicators of organizational capacity and program implementation.
Practical and research implications
Results contribute to a limited pool of research on investment in public health. Findings suggest an important and urgent need for investment in comprehensive heart health programs that is both sufficient and sustained. Attending to the sustainability of key staff positions is especially important. Findings also suggest that many competing priorities will challenge this investment, especially given the limited financial and human resources in public health (2% of the provincial health budget).
Consistent with others [cf. (Barry and Bialek, 2004; Lenaway et al., 2006)], this study also signals an important and urgent need for more research on investments in public health. As a first step, consistent data are needed on investments, including direct financial contributions and in-kind contributions from various sources. Another area of investigation is the cost of turnover related to orientation of new staff, discontinuities in relationships with community partners and loss of corporate memory. Appropriate financial expertise (e.g. accountants, economists) is needed to help provide estimates as part of routine data collection for comprehensive programs (Edwards, 2006). In the longer term, the use of common indicators across like programs would provide the necessary comparators.
|
CONCLUSION |
|---|
|
TOPSUMMARYINTRODUCTIONMETHODSRESULTSDISCUSSIONCONCLUSIONFUNDINGREFERENCES |
|---|
Investment decisions will continue to be made—with or without relevant evidence—especially given the urgency to take preventive action. To maximize the impact of limited resources, it is both a responsibility and an opportunity to generate and to apply ‘best available evidence’ related to investment in comprehensive public health programs. The most urgent and important needs include consistent and meaningful investment data across time and place, and knowledge of reasons to explain turnover of key staff positions that involve coordination of community partnerships.
|
FUNDING |
|---|
|
TOPSUMMARYINTRODUCTIONMETHODSRESULTSDISCUSSIONCONCLUSIONFUNDINGREFERENCES |
|---|
The evaluation was funded by the Ontario Ministry of Health and Long-Term Care. The views expressed in this paper do not necessarily represent the views of the Ontario government. Funding to pay the Open Access publication charges for this article was provided by the Centre for Behavioural Research and Program Evaluation.
|
ACKNOWLEDGEMENTS |
|---|
Thanks to Heather McGrath and Christine Herrera for their technical support in preparing the manuscript. Manuscript preparation was supported by a personnel award to Dr Riley from the Heart and Stroke Foundation of Canada and Institute of Circulatory and Respiratory Health, Canadian Institutes of Health Research, and a Nursing Chair to Dr Edwards funded by the Canadian Health Services Research Foundation, Canadian Institutes of Health Research and the Government of Ontario.
|
REFERENCES |
|---|
|
TOPSUMMARYINTRODUCTIONMETHODSRESULTSDISCUSSIONCONCLUSIONFUNDINGREFERENCES |
|---|
Aiken L. H., Clarke S. P., Sloane D. M., Sochalski J. A., Busse R., Clarke H., et al. Nurses' reports on hospital care in five countries. Health Affairs (2001) 20:43–53.
Ashley M. J., Boadway T., Cameron R., d'Avernas J. R., Ferrence R., Pipe A., et al. Actions Will Speak Louder Than Words: Getting Serious About Tobacco Control in Ontario. (1999) Toronto, Ontario: Ontario Tobacco Research Unit. A report to the Minister of Health from her Expert Panel on the renewal of the Ontario Tobacco Strategy (ISBN 0-96 86913-0-7).
Barry M., Bialek R. Tracking our investments in public health: what Have We Learned? Journal of Public Health Management and Practice (2004) 10:383–392.[Medline]
Canadian Heart Health Dissemination Research. International Union for Health Promotion and Education – Promotion & Education (2001) (Supplement 1).
Centres for Disease Control. Best Practices for Comprehensive Tobacco Control Programs. (1999) Atlanta, US: Department of Health and Human Services, CDC, National Centre for Chronic Disease Prevention and Health Promotion.
Champagne F., Leduc N., Denis J. L., Pineault R. Organizational and environmental determinants of the performance of public health units. Social Science and Medicine (1993) 37:85–95.[CrossRef][ISI]
Dunkley G., Michelin L. R., Stewart M. Towards Benchmarking Heart Health Coalitions: Developing a Systematic Process for Documenting and Enriching Community / Health Unit Partnerships. (2001) A collaborative initiative of Public Health Research, Education & Development Programs, the Association of Local Public Health Agencies, and the Ontario Council on Community Health Accreditation.
Edwards N. Investing in multi-level and multi-strategy disease and illness prevention. Community Health Research Unit (CHRU) Perspectives (2006) 13:1–2.
Germann K., Wilson D. Organizational capacity for community development in regional health authorities: A conceptual model. Health Promotion International (2004) 19:289–298.
Green L. W., Richard L., Potvin L. Ecological foundations of health promotion. American Journal of Health Promotion (1996) 10:270–281.[ISI][Medline]
Greenhalgh T., Robert G., Macfarlane F., Bate P., Kyriakidou O. Diffusion of innovations in service organizations: systematic review and recommendations. Milbank Memorial Fund (2004) 82:581–629.
Health Canada. Learning from SARS: Renewal of Public Health in Canada (2003) Health Canada, National Advisory Committee on SARS and Public Health. Ottawa, Ontario.
James L. R. Aggregation bias in estimates of perceptual agreement. Journal of Applied Psychology (1982) 67:219–229.[CrossRef][ISI]
Irvine D. M., Evans M. G. Job satisfaction and turnover among nurses: integrating research findings across studies. Nursing Research (1995) 44:246–253.[ISI][Medline]
Joffres C., Heath S., Farquharson J., Barkhouse K., Celeste L., MacLean D. Facilitators and challenges to organizational capacity building in heart health promotion. Qualitative Health Research (2004) 14:36–60.
Jones C. B. The costs of nurse turnover, part 2: application of the nursing turnover cost calculation methodology. Journal of Nursing Administration (2005) 35:41–49.[ISI][Medline]
Lafond A. K., Brown L., Macintyre K. Mapping capacity in the health sector: a conceptual framework. International Journal of Health Planning and Management (2002) 17:3–22.[CrossRef][ISI][Medline]
Larrabbee J. H., Janney M. A., Ostrow C. L., Withrow M. L., Gerald R., Burant C. Predicting registered nurse job satisfaction and intent to leave. Journal of Nursing Administration (2003) 33:271–283.[ISI][Medline]
Lenaway D., Halverson P., Sotnikov S., Tilson H., Corso L., Millington W. Public health systems research: setting a national agenda. American Journal of Public Health (2006) 96:410–413.
Lomas J. Connecting research and policy. Canadian Journal of Policy Research (2000) 1:140–144.
MacVicar N. Exploring the Role and Turnover among Heart Health Coordinators in the Ontario Heart Health Program: A Qualitative Study (2006) University of Ottawa. MSc.
Mays G. P., McHugh M. C., Shim K., Lenaway D., Halverson P. K., Moonesinghe R., et al. Getting what you pay for: public health spending and the performance of essential public health services. Journal of Public Health Management and Practice (2004) 10:435–443.[Medline]
Merzel C., D'Afflitti J. Reconsidering community-based health promotion: promise, performance, and potential. American Journal of Public Health (2003) 93:557–574.
Miller R. L., Shinn M. Learning from communities: overcoming difficulties in dissemination of prevention and promotion efforts. American Journal of Community Psychology (2005) 35:169–183.[CrossRef][ISI][Medline]
Mirolla M. The Cost of Chronic Disease in Canada (2004) The Chronic Disease Prevention Alliance of Canada (CDPAC).
Mittelmark M. B., Fosse E., Jones C., Davies M., Davies J. K. Mapping European capacity to engage in health promotion at the national level: HP-Source.net. International Union for Health Promotion and Education—Promotion & Education Supplement (2005) 1:33–39.
NSW Health. A Framework for Building Capacity to Improve Health (2001) Australia: Sydney.
Ontario Ministry of Health. Opportunities for Health: Promoting Heart Health. (1993) Toronto, Ontario. Report of the Chief Medical Officer of Health.
Ontario Tobacco Research Unit. Toward Ontario Tobacco Strategy Objectives 2003–2004 (2005) Toronto, Ontario: Ontario Tobacco Research Unit. [Special Reports: Monitoring and Evaluation Series, 2003–2004 (Vol. 10, Nos 1–4)].
Orlandi M. S. Health promotion technology transfer: organizational perspectives. Canadian Journal of Public Health (1996) 87:S28–S33.[ISI][Medline]
Riley B. Dissemination of heart health promotion in the Ontario Public Health System: 1989–1999. Health Education Research (2003) 18:15–31.
Riley B. L., d'Avernas J. R., Edwards N. C. Ontario Heart Health Program—Phase 1 (1998–2002): Final Evaluation Report. (2004) Ontario, Canada. Report prepared for the Ontario Ministry of Health and Long-term Care.
Riley B., Taylor S. M., Elliott S. J. Determinants of implementing heart health promotion activities in Ontario public health units: a social ecological perspective. Health Education Research (2001) 16:425–441.
Robinson K., Elliott S. J. Community development approaches to health promotion: a geographical perspective. Professional Geographer (1999) 51:283–295.[CrossRef]
Schabas R. Promoting heart health promotion. Canadian Journal of Public Health (1996) 87:S54–S56.[ISI][Medline]
Shields M. A., Ward M. Improving nurse retention in the National Health Service in England: the impact of job satisfaction on intentions to quit. Journal of Health Economics (2001) 20:677–701.[CrossRef][ISI][Medline]
Shrout P. E., Fleiss J. L. Intra class correlations: uses in assessing rater reliability. Psychological Bulletin (1979) 86:420–428.[CrossRef][ISI]
Steckler A., Goodman R. M., Alciati M. H. Collecting and analyzing organizational level data for health behavior research (Editorial). Health Education Research (1997) 12:i–iii.[ISI][Medline]
Strachota E., Normandin P., O'Brien N., Clary M., Krukow B. Reasons registered nurses leave or change employment status. Journal of Nursing Administration (2003) 33:111–117.[ISI][Medline]
Ulrich B. T., Buerhaus P. I., Donelan K., Norman L., Dittus R. How RNs view the work environment: results of a national survey of registered nurses. Journal of Nursing Administration (2005) 35:389–396.[ISI][Medline]
CiteULike 
Connotea 
Del.icio.us What's this?
| ||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||