Health Promotion International Advance Access originally published online on November 1, 2004
Health Promotion International 2004 19(4):489-500; doi:10.1093/heapro/dah411
| ||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||
HEALTH PROMOTION INTERNATIONAL Vol. 19. No. 4 © Oxford University Press 2004; All rights reserved.
PERSPECTIVES |
Program sustainability: focus on organizational routines
1Department of Social Studies of Medicine, McGill University, 3647 Peel Street, Montreal, Quebec, Canada, H3A 1X1, 2Groupe de Recherche Interdisciplinaire en Santé, University of Montreal, PO Box 6128, Downtown Station, Montreal, Quebec, Canada, H3C 3J7 and 3Régie Régionale de la Santé et des Services Sociaux du Bas-Saint-Laurent, Centre Hospitalier du Grand Portage, 75 St-Henri Street, Rivière-du-Loup, Quebec, Canada, G5R 2A4
Address for correspondence: Pierre Pluye, Department of Social Studies of Medicine, McGill University, 3647 Peel Street, Montreal, Quebec, Canada, H3A 1X1 E-mail: pierre.pluye{at}mail.mcgill.ca
 |
SUMMARY |
|---|
 TOP SUMMARY INTRODUCTIONLITERATURE REVIEWBACKGROUNDMETHODRESULTSDISCUSSIONREFERENCES |
|---|
Program sustainability is an ongoing concern for most people in health promotion. However, the current notion of sustainability in organizations, namely routinization, needs refinement. This article examines organizational routines. In so doing, it refines the notion of sustainability and the assessment of routines. Drawing on the organizational literature, a routinized program is defined by the presence of routinized activities, meaning that these activities exhibit four characteristics of organizational routines: memory, adaptation, values and rules. To answer the question of how these characteristics are useful, we conducted an empirical study of the routinization of the Quebec Heart Health Demonstration Project in five community health centers. Our method consisted of a multiple-case study. We observed project activities in each center in 2000. The data came from documents and interviews with project actors. Our results show that, in one of the centers, no resources had been officially committed to project activities. Even so, the actors continued some activities on an informal basis. In another center, the activities satisfied three of the four routine characteristics. In the three others, activities satisfied all of the characteristics. These results suggest focusing the study of program sustainability on the routinization of activities resulting from it. They indicate four distinct degrees of sustainability: (1) the absence of sustainability; no program activity is continued; (2) precarious sustainability; some residual activities are pursued, at least unofficially; (3) weak sustainability; the program produces some official activities that are not routinized; and (4) sustainability through routinization; routinized activities result from the program.
Key words: program sustainability; organizational routines; organizational learning
|
INTRODUCTION |
|---|
|
TOPSUMMARYINTRODUCTIONLITERATURE REVIEWBACKGROUNDMETHODRESULTSDISCUSSIONREFERENCES |
|---|
People involved in programs want to know if "their" program will endure (Altman et al., 1991
). In health promotion, the notion of sustainability refers to the continuation of programs. However, this notion needs to be refined. Sustainability is linked to four important issues in health promotion. First, sustainability maintains programs' effects over a long period (Puska et al., 1996; Manfredi et al., 2001). Second, many programs aim at behavioral changes, and they must survive over an extended period of time for such changes to occur (Steckler and Goodman, 1989). Third, there is often a latent period of years between when programs begin and when their effects on population health are felt (Roussos and Fawcett, 2000). Fourth, organizations and actors lose what they have invested when programs are not sustainable, and they will resist further investment (Steckler and Goodman, 1989; Rissel et al., 1995; Shediac-Rizkallah and Bone, 1998). |
LITERATURE REVIEW |
|---|
|
TOPSUMMARYINTRODUCTIONLITERATURE REVIEWBACKGROUNDMETHODRESULTSDISCUSSIONREFERENCES |
|---|
The literature suggests that routinization is the primary or fundamental process in the sustainability of health promotion programs (Yin, 1979
; Steckler and Goodman, 1989; Weber, 1995; Shediac-Rizkallah and Bone, 1998; Pluye et al., 2004a). Routinization refers to sustainability in organizations. The pioneering contribution of Goodman et al. (Goodman et al., 1993) sets out to operationalize the concept of routinization and to propose a measurement. However, their proposition raises some concerns (Scheirer, 1993). In our opinion, organizational routines and the process of routinization, as it has occurred in the past, leading to the routines currently in place, are two different issues (Pluye et al., 2004a). The present paper aims to assess the presence of organizational routines.
We define "program" as being a set of activities aimed at achieving an objective (Nancholas, 1998). According to Weber (Weber, 1995), program objectives define activities. Weber conceives activities as the understandable orientation of the behavior of people in relation to the behavior of other people. Explicit program objectives make this orientation manifest and therefore understandable. Given this link between program objectives and activities, we believe that programs are routinized in organizations when objective-related activities are routinized.
Yin (Yin, 1979) defines routines as organizational activities for which sustainable resources are mobilized. This means that: (1) the financing of routinized activities is an integral part of an organization's regular budget; (2) the personnel in charge of routines hold permanent positions, and these activities are subject to formal task descriptions; and (3) materials required for completing routines appear on an organization's inventory.
Referring to Katz and Kahn (Katz and Kahn, 1978), Goodman et al. (Goodman et al., 1993) define routines as official activities in relation to four organizational functions. (1) Production: objectives of routinized activities are part of an organization's plans. (2) Maintenance: these activities are carried out by regular employees and are backed by an organization's management. (3) Support: they benefit from stable financing and materials. (4) Management: they are formally supervised in organizations and are subject to written task descriptions.
Goodman et al. (Goodman et al., 1993) have measured the routinization of health promotion programs in 141 organizations using these functions. They propose three to five questionnaire items to assess each function. Each item queries the number of years since the routine's element in question has been in place. However, results suggest this measure is not linked to managers' perception of routinization (Goodman et al., 1993). This is to be expected. The time elapsed appears to introduce a bias (Scheirer, 1993). Organizational routines and the process of routinization are two different issues. The more routinized the program, the more likely its activities have been going on for some time. Conversely, length of time gives no indication of the program's present and future; the time elapsed since the activities began is not related to the presence of routines and does not predict routinization.
Like Goodman et al. (Goodman et al., 1993), we believe that the study of organizational routines plays a key role in the examination of program sustainability (Pluye et al., 2000; Pluye et al., 2004a). In order to define routines, we have reviewed organizational studies. According to Cyert and March (Cyert and March, 1970), routines consist of collective procedural actions that have a natural tendency to be perpetuated. The literature on organizational learning examines the evolution of routines and offers an authoritative framework to complete this definition (Argyris and Schön, 1978; Hedberg, 1981; Huber, 1996; Levitt and March, 1996; Weick, 1996; Edmondson and Moingeon, 1998; Argote, 1999). This literature suggests four characteristics of organizational routines: memory, adaptation, values and rules.
Memory
Routines become memorized in organizations (Argyris and Schön, 1978; Walsh and Ungson, 1991; Cohen and Bacdayan, 1996; Cook and Yanow, 1996). Organizational memory is defined as shared interpretations of past experiences that are brought to bear on present activities (Stein, 1995) and has three major components: social networks, paper-based manuals and computerized memory (Huber, 1996). This memory requires stable resources. For example, social networks are vulnerable to a high turnover of actors (Stein, 1995; Carley, 1996; Girod-Seville, 1996; Argote, 1999).
Adaptation
Routinized activities are adapted in accordance with their context (Cyert and March, 1970; Hutchins, 1996). For example, a study of the hiring procedures in residence halls of a university shows how these organizational routines were simplified (adapted to the multiplicity of halls), while standard elements remained unchanged (Feldman, 2000). Conversely, this adaptation can sometimes be questioned, and resulting routinized activities may be mis-adapted. These activities have few or detrimental effects, constitute defensive routines and derive from superstitious learning (Argyris, 1993; Levitt and March, 1996; Edmondson and Moingeon, 1998).
Values
Routinized activities in organizations reflect collective values and beliefs. These values and beliefs define what is good, aesthetic and true, and are concretized when objectives are formulated (Cyert and March, 1970). They are also manifested in cultural artifacts such as codes, symbols, rituals or jargon. People create inter-subjective meanings that are expressed in and through these artifacts (Cook and Yanow, 1996).
Rules
Finally, routinized activities conform to rules governing decision-making and action (Cyert and March, 1970; Dogson, 1993; Cohen and Bacdayan, 1996; Hutchins, 1996). In every organization, these rules account for "the way things are done around here" (Levitt and March, 1996, p. 525).
|
BACKGROUND |
|---|
|
TOPSUMMARYINTRODUCTIONLITERATURE REVIEWBACKGROUNDMETHODRESULTSDISCUSSIONREFERENCES |
|---|
In order to explore how these four characteristics operationalize organizational routines, we conducted an empirical study of the routinization of the Quebec Heart Health Demonstration Project, hereinafter called "the project". We examined five community health centers in 2000 and their activities as they related to the project (Potvin et al., 1992
; Pelletier et al., 1997). The history of heart health promotion at each site is presented in Table 1. Between 1988 and 1991, the five community health centers introduced a community action project for heart health promotion.
|
The core of the project's activities consisted of providing support to 12 volunteer committees (Table 1). The specific objective of this article is to examine the activities resulting from this project in each center according to the proposed characteristics of routines. We wanted to answer the question of how these characteristics were useful for understanding project routinization in the five centers.
|
METHOD |
|---|
|
TOPSUMMARYINTRODUCTIONLITERATURE REVIEWBACKGROUNDMETHODRESULTSDISCUSSIONREFERENCES |
|---|
The method is a multiple-case study (Yin, 1994
). Each center constitutes a case. For each center, we examined activities resulting from the project in 2000. Two sources of data were used: documentation pertaining to the project and interviews held with actors in the centers. These actors were community organizers, nurses and physicians who were all in charge of the project and its activities. As usual, in the development of community coalitions, these professionals initiated the formation of heart health committees. Relationships between these actors, community volunteers and the regional health authority are described elsewhere (Pluye et al., 2004b). The documentation allowed us to list activities pertaining to the project in each center, notably those completed in 1999. At the start of each interview, the actors reviewed and updated these lists for the current year. The interviews consisted of 15 questions exploring how the activities completed in 2000 were characterized in terms of memory, adaptation, values and rules.
These characteristics are grounded in a literature that focuses on routine and does not contradict organizational functions from which the items proposed by Goodman et al. were derived (Goodman et al., 1993). These items were conceived with an expert panel and were used in at least three empirical studies (Goodman et al., 1993; Barab et al., 1998; Goodson et al., 2001). Thus, we chose to formulate our questions based on these items. Eleven out of fifteen questions were modifications of these items (Figure 1). Given the bias mentioned, these modifications consisted mainly of extracting time reference from the items. For example, the question "is a supervisor formally assigned to the activities?" came from the item "number of years supervisor formally assigned?" Implicitly, Goodson et al. also modified these items by eliminating any temporal reference (Goodson et al., 2001).
|
To operationalize the four routine characteristics, we proceeded as follows. First, organizational routine memory needs stable resources; therefore memory-related questions were modified from the items pertaining to the stability of human, financial and material resources (support and maintenance functions). Second, two adaptation-related questions came from the items about evaluation and strategies (production and management functions). Moreover, given that Goodman et al. did not examine the presence of mis-adapted routines, we formulated an original question on this topic (Goodman et al., 1993
). Third, a value-related question was modified from the item about program objectives (production function). However, the study of Goodman et al. did not focus on values, and we formulated three questions referring to this characteristic (Goodman et al., 1993). Finally, rules-related questions were modified from the items about supervision and documents (production and management functions). The project's regional coordinator assisted with the data collection. He provided the entire project documentation and the list of actors involved in the project between 1987 and 2000. Among the 23 actors contacted, 12 were familiar with the activities in 2000 and were interviewed (two or three from each center). Interviews lasted between 15 and 30 minutes. For interviews about Centers A, B, C and D, all 15 questions were asked. Center E's interviewees stated that no center resources were officially committed to activities resulting from the project, and therefore the questions concerning adaptation, values and rules were dropped from these interviews. Interviews were recorded on audiocassette and transcribed.
For each center, transcripts were aggregated in a synopsis. Each synopsis lists activities and answers to the questions. For each question, the factual content of transcripts was reworded in readable, short sentences coded according to the presence/absence of qualitative evidence for one activity or more. For example, a Center A-related transcript statement "A community organizer two hours a week because the committees are very autonomous" was reworded as "The community organizer worked two hours a week for the activities" (Pluye, 2002, p. lix) and coded "A community organizer two hours a week" (Table 3). For each characteristic of routines, results derived from binary coding (presence/absence of qualitative evidence for one characteristic-related question or more). Synopses and coding were first reviewed by a key informant in a face-to-face interview, i.e. the project's regional coordinator, and were then reviewed by the actors during telephone interviews. Finally, results from the five centers were compared.
|
|
RESULTS |
|---|
|
TOPSUMMARYINTRODUCTIONLITERATURE REVIEWBACKGROUNDMETHODRESULTSDISCUSSIONREFERENCES |
|---|
Our inventory of heart health activities conducted in 2000 in the five centers is presented in Table 2. This table shows that heart health activities were of two types: local development and social planning. The former are those that support local committees and the latter are those directly completed by centers' staff. In addition, Table 2 shows that there were official heart health activities in four of the five centers in 2000.
|
Center E was no longer officially supporting any activity related to the project. No resources were officially committed. Nonetheless, three actors still maintained unofficial activities derived from the project, even though the center's management was unaware of this fact. First, an actor tested for high blood pressure while administering flu vaccinations to elderly people. "It started with the project because I was the one who planned the vaccinations; I was the one who took this initiative, and it has continued." Second, another actor organized conferences for community organizations and used them as a forum to continue delivering heart health education. "It is not part of a program because it is really at a personal level that I am approached." Third, another actor was involved in the promotion of physical activity in community organizations. "I do not have official support for (this work); I promote physical activity in community organizations, but the center doesn't officially recognize it, at least not on paper."
Centers A, B, C and D continue to support activities related to the project. These activities were similar in terms of three routine characteristics in the four centers (memory, adaptation and values). Centers A, B and C nevertheless differed from Center D with respect to the fourth characteristic, rules (Table 3).
Memory
The financial resources committed to activities were an integral part of budgets at Centers A, B, C and D. The human and material resources amounted to several hours of work per week and only a hundred dollars per year.
Adaptation
We found adaptations of activities in Centers A, B, C and D. Activities that volunteers and the public asked for most often were renewed yearly, and those that did not provoke much response were not repeated. Local development activities were adapted according to feedback from volunteers and volunteers' availability. As one actor put it, "most of the activities are conducted on demand; for example, small scale community conferences are organized by request, or according to the general level of interest; we meet with the volunteers before their first activity of the year, and we ask them about their expectations, needs and the amount of time they feel like investing." Social planning activities, on the other hand, were a service offered by centers C and D, and they were adapted to their clients' requests. For example, the number of smoking cessation courses given annually was a function of the number of people interested in attending.
Mis-adapted activities were also observed in centers A, B, C and D. According to the actors, at least one activity in each center was conducted because it responded to volunteer interest, despite actors' skepticism about its relevance. (1) In Center A, supporting educational activities like community conferences did not affect the prevalence of risk factors as one might have expected. An actor was concerned that the same small group of people was always participating in these activities; therefore, heart health messages were diffused to only a small group that already had the information. (2) In Center B, efforts to sensitize young people to the risks of smoking did not yield expected results on the target audience. "Undertaking this (annual) activity without follow-up is a complete waste of time; as professionals, we know this very well, but we cannot really tell the volunteers not to do it, basically because we like the idea that they go into the schools and talk about smoking." (3) In Center C, supporting community blood pressure screening did not reduce the number of people with high blood pressure. "We had the impression that people came in order to check their pressure because they already knew what it was; we doubt the usefulness of the screenings, or at least we wonder how useful they are, because we have not met our original objective, which was to find new cases of high blood pressure in the community; people nevertheless enjoy these screenings." (4) In Center D, actors also raised doubts about the relevance of such screenings.
Values
Activities corresponded to the explicit objectives of Centers A, B, C and D. Plans were revised annually at Centers A and C, and objectives were updated. Center B's plan had not been revised for two or three years. Actors at Center D worked together to set their objectives without any contact with the Center's managers. "We are left to our own devices; we prepare the annual plan together on our own." Activities in Centers A, B, C and D were associated with symbols but not with any rituals. The actors in Center D had a breakfast meeting with committee volunteers for Christmas, but these meetings were only organized from time to time, and did not become an annual ritual. In addition, activities at Centers A, B, C and D had their own jargon. For example, the actors and volunteers spoke of a heart health promotion activity dealing with nutrition as a "food booth".
Rules
Activities at Centers A, B and C were subject to rules, but this was not the case at Center D. Activities were supervised by a coordinator in Centers A, B and C, but not in Center D. According to one actor in Center D, "there was one key person, but the responsibilities were never officially recognized; heart health activities have not really been entrenched because we do not have control over them; the Center's management lets us do what we want, there is no supervision." In Centers A, B and C, the activities and a description of the tasks were integrated into the planning process, but this was not the case at Center D. As a Center D actor put it, "I don't know if heart health is mentioned as one of our activities somewhere in the Center's documentation, but I don't think so." Finally, written rules corresponded to activities in Centers A, B and C. There were procedural manuals for cardiovascular risk factors screening, protocols for community-based cardiac rehabilitation and heart health promotion manuals dealing with healthy eating. Center D had no such written rules.
|
DISCUSSION |
|---|
|
TOPSUMMARYINTRODUCTIONLITERATURE REVIEWBACKGROUNDMETHODRESULTSDISCUSSIONREFERENCES |
|---|
Results indicate that the activities in 2000 varied by center and according to the characteristics of routines. The project was routinized in Centers A, B and C, and not routinized in Centers D and E. No routinized activities in Center E were originated from the project. Remaining activities in Center D have not been routinized; they do not meet all the characteristics. Activities in Centers A, B and C were routinized with all the characteristics of routines. These results support the notion that a simple analysis of current activities according to the four proposed characteristics of organizational routines enables the assessment of the routinization of health promotion programs.
Results from Center E showed the absence of any activity resulting from the project. The literature on health promotion provides several examples where no long-term activities were continued (Goodman and Steckler, 1988; Florin et al., 1992; Rissel et al., 1995; O'Loughlin et al., 1998; Shediac-Rizkallah and Bone, 1998; Wickizer et al., 1998; Lackey et al., 2000).
Furthermore, results from Center E suggest something that had not been described previously; namely the existence of unofficial activities originating from programs. In Weber's terms, these activities consist of behavior for which the intent is understood by some people but not by Center E's managers, who are simply unaware of them. We had expected that some of the centers' activities would be routinized and others not, but we had not expected to see them running outside of managerial awareness.
This observation of unofficial activities was implicit in two other studies. Wickizer et al. looked at the sustainability of 11 programs financed by the Kaiser Foundation (Wickizer et al., 1998, p. 138). One program was terminated but "selected prevention activities were continued; limited school-based activities and community parenting classes were maintained on an informal basis". Lichtenstein et al. examined the sustainability of the COMMIT program (Lichtenstein et al., 1996). They found that some of COMMIT's smoking prevention activities were continued without ongoing financing or paid staff.
This analysis of the literature allows the following interpretation of our results. The absence of official activities in Center E suggests a precarious level of sustainability, taking the form of a few unofficial activities derived from the project. The presence of remaining activities in Center D suggests a weak level of sustainability because of the lack of any routinized activities. Finally, the presence of routinized activities in Centers A, B and C suggests the project is well sustained in these centers.
Speaking more generally, these results suggest four degrees of program sustainability in organizations. (1) The absence of sustainability. The program is not sustained; no ongoing activity comes out of it. (2) Precarious sustainability. The program is sustained, but the future of its status is uncertain. Actors maintain some residual activities on an informal basis as part of their functions in the organization, but this is completely unrelated to the program. The continuation of these activities depends entirely on the initiative of these actors. (3) Weak sustainability. The program is sustained but remaining activities are weakly maintained. Official activities result but they are not routinized. These activities may be subject to radical changes in the short term. (4) Sustainability through routinization. The program is sustained, activities have resulted from it, and they have been routinized. Their maintenance seems assured in the long term. In organizations under study, different sets of activities (defining the program) were undertaken towards the heart health objective. In the present paper, those sets constitute the unit of analysis, and we assessed the routinization of these sets at the organizational level. At this level of analysis, the four degrees of sustainability constitute an ordinal scale.
Our method appears to be simple and valid. First, this method is based on documents that list activities and on a shared understanding of which activities reflected the project in centers. For each center, a 15-question interview with key actors was used to collect data about these activities. For each question and characteristic, data were coded according to the presence/absence of qualitative evidence. Results were validated by the project's actors and a key informant. Second, the small sample size (five organizations) does not appear to have affected the validity of the results, as shown by the variation in observations by center and according to the characteristics of routines.
Our study nevertheless faces a limitation generally associated with researches on sustainability (Green, 1989). Results make no judgment about the relevance of the continued or routinized activities, particularly concerning their relationship to program objectives. Our conception of routines also includes mis-adapted activities. Program stakeholders and decision-makers should be wary of the existence of such mis-adapted activities. Finding them is justification enough for completing an evaluation. Thus, the search for these activities may contribute to the issue of evaluating both quality and sustainability (Rissel et al., 1995). In other words, these results lead us to believe that studying program sustainability provides an opportunity for a critical review of program activities as described by Argyris and Schön (Argyris and Schön, 1999).
In this article, we examine current routines. Excluding the temporal dimension of the questions adapted from Goodman et al. (Goodman et al., 1993) is a clear break from their propositions on the measurement of program routinization. Our results describe routinized activities at a specific time, without taking into account previous routinization processes that have led to current routines. This removal of the temporal dimension of the questions was done implicitly in one other study (Goodson et al., 2001). This removal addresses the mentioned bias, and routinization processes are examined elsewhere (Pluye et al., 2004a).
The present article offers two more improvements on previous works. First, we suggest that the assessment of the existence of routine is a prerequisite for the further examination of routines, in particular their extent in organizations as proposed by Goodman et al. (Goodman et al., 1993). In our opinion, measuring the extent of non-routinized activities according to their "optimum limits" (Goodman et al., 1993, p. 166) appears to make little sense, e.g. the extent of unofficial activities. Second, our work is grounded in the literature focused on organizational routines. This literature refines the general functions of organizations proposed by Katz and Kahn (Katz and Kahn, 1978) that inspired Goodman et al. (Goodman et al., 1993), and allows us to better assess the existence of organizational routines, e.g. by taking their cultural characteristics into account.
In conclusion, we refine the basis for assessing program sustainability. The refinement of the concept of organizational routines seems to indicate four degrees of sustainability: the absence of any activity derived from programs, the presence of unofficial activities, the presence of remaining official activities, and the presence of routinized activities. This ordinal scale must be further validated in studies with larger samples.
|
ACKNOWLEDGEMENTS |
|---|
The authors wish to thank Daniel Campeau for his comments on previous versions of this paper. Pierre Pluye holds a post-doctoral fellowship grant from the Institute of Health Services and Policy Research – Canadian Institutes of Health Research – (IHSPR-CIHR # 555–84). Louise Potvin holds the Chair in Community Approaches and Health Inequalities funded by the Canadian Health Services Research Foundation and the Canadian Institutes of Health Research (CHSRF-CIHR # CPI-052605). Jean-Louis Denis holds the Chair in Transformation and Governance of Health Care Organizations (CHSRF-CIHR # CPI-052505).
|
REFERENCES |
|---|
|
TOPSUMMARYINTRODUCTIONLITERATURE REVIEWBACKGROUNDMETHODRESULTSDISCUSSIONREFERENCES |
|---|
Altman, D. G., Endres, J., Linzer, J., Lorig, K., Howard-Pitney, B. and Rogers, T. (1991) Obstacles to and future goals of ten comprehensive community health promotion projects. Journal of Community Health, 16, 299–314.[CrossRef][Medline]
Argote, L. (1999) Organizational Learning: Creating, Retaining and Transferring Knowledge. Kluwer, Boston.
Argyris, C. (1993) Knowledge for Action. Jossey-Bass, San Francisco.
Argyris, C. and Schön, D. A. (1978) Organizational Learning: A Theory of Action Perspective. Addison-Wesley Publishing Company, London.
Argyris, C. and Schön, D. A. (1999) Théorie et Pratique Professionnelle: Comment en Accroître l'Efficacité. Les Éditions Logiques, Outremont.
Barab, S. A., Redman, B. K. and Froman, R. D. (1998) Measurement characteristics of the levels of institutionalization scales: examining reliability and validity. Journal of Nursing Measurement, 6, 19–33.[Medline]
Carley, K. (1996) Organizational learning and personal turnover. In Cohen, M. D. and Sproull, L. S. (eds) Organizational Learning. Sage, Thousand Oaks, pp. 230–266.
Cohen, M. D. and Bacdayan, P. (1996) Organizational routines are stored as procedural memory: evidence from a laboratory study. In Cohen. M. D. and Sproull, L. S. (eds) Organizational Learning. Sage, Thousand Oaks, pp. 403–429.
Cook, S. D. N. and Yanow, D. (1996) Culture and organizational learning. In Cohen, M. D. and Sproull, L. S. (eds) Organizational Learning. Sage, Thousand Oaks, pp. 430–459.
Cyert, R. M. and March, J. G. (1970) Processus de Décision Dans l'Entreprise. Dunod, Paris.
Dogson, M. (1993) Organizational learning: a review of some literature. Organization Studies, 14, 375–394.
Edmondson, A. and Moingeon, B. (1998) From organizational learning to the learning organization. Management Learning, 29, 5–20.
Feldman, M. (2000) Organizational routines as a source of continuous change. Organization Science, 11, 611–629.
Florin, P., Chavis, D., Wandersman, A. and Rich, R. (1992) A system approach to understanding and enhancing grassroots organizations: The Block Booster Project. In Levine, R. L. and Fitzgerald, H. E. (eds) Analysis of Dynamics Psychological Systems: Methods and Applications. Plenum Press, New York, pp. 215–243.
Girod-Séville, M. (1996) La Mémoire des Organisations. L'Harmattan, Paris.
Goodman, R. M., McLeroy, K. R., Steckler, A. B. and Hoyle, R. H. (1993) Development of level of institutionalization scales for health promotion programs. Health Education Quarterly, 20, 161–178.[Web of Science][Medline]
Goodman, R. M. and Steckler, A. B. (1988) The life and death of a health promotion program: an institutionalization case study. International Quarterly of Community Health Education, 8, 5–21.
Goodson, P., Murphy Smith, M., Evans, A., Meyer, B. and Gottlieb, N. H. (2001) Maintaining prevention in practice: survival of PPIP in primary care settings. American Journal of Preventive Medicine, 20, 184–189.[CrossRef][Web of Science][Medline]
Green, L. W. (1989). Is institutionalization the proper goal of grantmaking? American Journal of Health Promotion, 3, 44.
Hedberg, B. (1981) How organizations learn and unlearn. In Nystrom, P. C. and Starbuck, W. H. (eds) Handbook of Organizational Design. Oxford University Press, Oxford, pp. 3–27.
Huber, G. P. (1996) Organizational learning: the contributing processes and the literatures. In Cohen, M. D. and Sproull, L. S. (eds) Organizational Learning. Sage, Thousand Oaks, pp. 124–162.
Hutchins, E. (1996) Organizing work by adaptation. In Cohen, M. D. and Sproull, L. S. (eds) Organizational Learning. Sage, Thousand Oaks, pp. 20–57.
Katz, D. and Kahn, R. L. (1978) The Social Psychology of Organizations. John Whiley & Sons, New York.
Lackey, J. F., Welnetz, T. and Balistrieri, M. (2000) Ensuring a prevention presence when coalitions falter: a case study of a community coalitions demonstration project. Evaluation and Program Planning, 23, 355–363.
Levitt, B. and March, J. G. (1996) Organizational learning. In Cohen, M. D. and Sproull, L. S. (eds) Organizational Learning. Sage, Thousand Oaks, pp. 516–540.
Lichtenstein, E., Thompson, B., Nettekoven, L. and Corbett, K. (1996) Durability of tobacco control activities in 11 North American communities: life after the community intervention trial for smoking cessation (COMMIT). Health Education Research, 11, 527–534.
Manfredi, C., Crittenden, K., Ik Cho, Y., Engler, J. and Warnecke, R. (2001) Maintenance of a smoking cessation program in public health clinics beyond the experimental evaluation period. Public Health Reports, 116, 120–135.[CrossRef][Web of Science][Medline]
Nancholas, S. (1998) A logical framework. Health Policy and Planning, 13, 189–193.
Pelletier, J., Moisan, J., Roussel, R. and Gilbert, M. (1997) Heart health promotion: a community development experiment in a rural area of Québec, Canada. Health Promotion International, 12, 291–298.
Pluye, P., Potvin, L. and Denis, J. L. (2000) La pérennisation des projets pilotes en promotion de la santé. Ruptures, Revue Transdisciplinaire en Santé, 7, 99–113.
Pluye, P. (2002) Vers un nouveau modèle théorique du déroulement des programmes: Étude de la routinisation des programmes en promotion de la santé. Ph.D. Thesis, Report Number T02-04, Interdisciplinary Health Research Group (GRIS), Faculty of Medicine, University of Montreal, Montreal.
Pluye, P., Potvin, L. and Denis, J. L. (2004a) Making programs last: conceptualizing sustainability. Evaluation and Program Planning, 27, 121–133.[CrossRef][Web of Science]
Pluye, P., Potvin, L. and Pelletier, J. (2004b) Community coalitions and health promotion: is it that important to develop or not an interorganisational network? Promotion and Education, 11, 17–23.
Potvin, L., Paradis, G., Laurier, D., Masson, P., Pelletier, J. and Lessard, R. (1992) Le cadre d'intervention du Projet Québécois de Démonstration en Santé du Coeur. Hygie, 11, 17–23.[Medline]
Puska, P., Nissinen, A., Tuomilehto, J., Salonen, J. T., Koskela, K., McAlister, A., et al. (1996) In Pan American Health Organization (ed.) Health Promotion: An Anthology. Pan American Health Organization, Washington, pp. 89–125.
Rissel, C., Finnegan, J. and Bracht, N. (1995) Evaluating quality and sustainability: issues and insights from the Minnesota Heart Health Program. Health Promotion International, 10, 199–207.
Roussos, S. T. and Fawcett, S. B. (2000) A review of collaborative partnerships as a strategy for improving community health. Annual Review of Public Health, 21, 369–402.[CrossRef][Web of Science][Medline]
Scheirer, M. A. (1993) Are the level of institutionalization scales ready for "prime time"? A commentary on "development of level of institutionalization scales for health promotion programs". Health Education Quarterly, 20, 179–183.[Web of Science][Medline]
Shediac-Rizkallah, M. C. and Bone, L. R. (1998) Planning for the sustainability of community-based health programs: conceptual frameworks and future directions for research, practice and policy. Health Education Research, 13, 87–108.
Steckler, A. and Goodman, R. M. (1989) How to institutionalize health promotion programs. American Journal of Health Promotion, 3, 34–44.
Stein, E. W. (1995) Organizational memory: review of concepts and recommendations for management. International Journal of Information Management, 15, 17–33.
Walsh, J. P. and Ungson, G. R. (1991) Organizational memory. The Academy of Management Review, 16, 57–91.
Weber, M. (1995) Économie et Société: Les Catégories de la Sociologie. Plon, Paris.
Weick, K. E. (1996) The role of renewal in organizational learning. International Journal of Technology Management, 11, 738–746.
Wickizer, T. M., Wagner, E., Cheadle, A., Pearson, D., Beery, W., Maeser, J., et al. (1998) Implementation of the Henry J. Kaiser family foundation's community health promotion grant program: a process evaluation. The Milbank Quarterly, 76, 121–147.[CrossRef][Web of Science][Medline]
Yin, R. K. (1979) Changing Urban Bureaucracies: How New Practices Become Routinized. Lexington Books, Lexington.
Yin, R. K. (1994) Case Study Research: Design and Methods. Sage, Thousand Oaks.
CiteULike 
Connotea 
Del.icio.us What's this?
This article has been cited by other articles:
|
|
 |
|
  R. Savaya, G. Elsworth, and P. Rogers Projected Sustainability of Innovative Social Programs Eval Rev, April 1, 2009; 33(2): 189 - 205. [Abstract] [PDF]
|
|
|
|
 |
|
 R. Savaya, S. Spiro, and R. Elran-Barak Sustainability of Social Programs: A Comparative Case Study Analysis American Journal of Evaluation, December 1, 2008; 29(4): 478 - 493. [Abstract] [PDF]
|
|
|
|
 |
|
 M. Jansen, J. Harting, N. Ebben, B. Kroon, J. Stappers, E. Van Engelshoven, and N. de Vries The concept of sustainability and the use of outcome indicators. A case study to continue a successful health counselling intervention Fam. Pract., December 1, 2008; 25(suppl_1): i32 - i37. [Abstract] [Full Text] [PDF]
|
|
|
|
 |
|
 F. Baum, G. Jolley, R. Hicks, K. Saint, and S. Parker What makes for sustainable Healthy Cities initiatives?--a review of the evidence from Noarlunga, Australia after 18 years Health Promot. Int., December 1, 2006; 21(4): 259 - 265. [Abstract] [Full Text] [PDF]
|
|
|
|
 |
|
 L. Potvin, S. Gendron, A. Bilodeau, and P. Chabot Integrating Social Theory Into Public Health Practice Am J Public Health, April 1, 2005; 95(4): 591 - 595. [Abstract] [Full Text] [PDF]
|
|
| ||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||