Health Promotion International Advance Access originally published online on June 2, 2006
Health Promotion International 2006 21(3):245-255; doi:10.1093/heapro/dal017
| ||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||
DEBATE |
Assessing delivery of the five ‘As’ for patient-centered counseling
Clinical Research Unit, Kaiser Permanente Colorado Denver, CO, USA 1 White Mountain Research Associates, L.L.C. Danbury, NH, USA 2 Rush Medical College Chicago, IL, USA
Address for correspondence: Russell E. Glasgow, PhD, Clinical Research Unit, Kaiser Permanente Colorado, 335 Road Runner Lane, Penrose, CO 81240, USA E-mail: russg{at}ris.net
 |
SUMMARY |
|---|
 TOP SUMMARY INTRODUCTIONCONCLUSIONSREFERENCES |
|---|
The ‘5As’ model of behavior change provides a sequence of evidence-based clinician and office practice behaviors (Assess, Advise, Agree, Assist, Arrange) that can be applied in primary care settings to address a broad range of behaviors and health conditions. Although the 5As approach is becoming more widely adopted as a strategy for health behavior change counseling, practical and standardized assessments of 5As delivery are not widely available. This article provides clinicians and researchers with alternatives for assessment of 5As implementation for both quality improvement, and for research and evaluation purposes, and presents several practical tools they may wish to use. Sample instruments for tracking delivery of the 5As and related tools that are in the public domain are provided to facilitate integration of self-management support into clinical care. We discuss the strengths and limitations of the various assessment approaches. Promising and practical measures to assess the 5As exist for both quality improvement and research purposes. Additional validation is needed on almost all current procedures, and both clinicians and researchers are encouraged to use these instruments and share the resulting data.
Key words: assessment; primary care; behavior change
|
INTRODUCTION |
|---|
|
TOPSUMMARYINTRODUCTIONCONCLUSIONSREFERENCES |
|---|
The ‘5As’ model of behavior change counseling is an evidence-based approach appropriate for a broad range of different behaviors and health conditions, and is feasible to apply in primary care (Fiore et al., 2000
; Glasgow et al., 2001a; Glasgow et al., 2002; Serdula et al., 2003; Glasgow et al., 2004b; Goldstein et al., 2004; The Quality Indicator Study Group, 1995). The 5As are as follows: assessing patient level of behavior, beliefs and motivation; advising the patient based upon personal health risks; agreeing with the patient on a realistic set of goals; assisting to anticipate barriers and develop a specific action plan; and arranging follow-up support (Moen et al., 1999; Glasgow et al., 2002; Berwick, 2003; Glasgow et al., 2003). However, there are no standard, widely used assessments of 5As delivery. Most studies of the 5As have developed their own measures. As the 5As become more widely adopted, it is increasingly important to have reliable, valid and practical measures of these strategies.
Recently, the Lifescripts consortium, through funding by the Australian Government Department of Health and Ageing, released a series of practice tools and other resources to support general practitioners in reducing lifestyle risk factors among their patients (National Heart Foundation of Australia and Kinect Australia for the Lifescripts Consortium, 2005). The Lifescripts package contains a variety of evidence-based resources, manuals and user-friendly assessment tools for GPs, practice nurses and staff in the general practice setting. The program and associated assessment tools build on the SNAP framework developed by the Royal Australian College of General Practitioners (Smoking, Nutrition, Alcohol and Physical activity) (The Royal Australian College of General Practitioners, 2004). This framework utilizes the 5As for incorporating effective clinical strategies for the detection, assessment and management of SNAP risk factors in general practice settings.
Documentation and tracking of the 5As as a way of delivering self-management support is now required by the Joint Commission for the Accreditation of Health Care Organizations as part of its Disease Management Certification program. Health care organizations satisfying this requirement as part of their disease management protocols are more likely to receive certification. This certification could provide health care organizations with leverage to negotiate higher reimbursement rates from third-party payers.
The purposes of this article are to (i) discuss assessment of the 5As from both quality improvement (QI) and research perspectives, (ii) describe and provide examples of instruments that have been used to assess 5As implementation and (iii) discuss the strengths and limitations of present 5As assessment approaches.
There are four primary sources of information that can be used to assess delivery of the 5As as follows: direct observation of patient–clinician encounters; medical records; clinician reports; and patient reports (Table 1). Each has its strengths and limitations, and is used for different purposes. Multiple approaches were used to identify potential instruments as follows: (i) an electronic search was conducted using PubMed with various combinations of the keywords ‘5As, behavioral counseling, assessment and physician advice’; (ii) recent articles and reviews of behavioral counseling that included the 5As were consulted and reference lists in these articles were searched; and (iii) contacts were made with leading researchers in the area. Given that there are no MeSH headings related to the 5As and the recency of much of the work in this area, the latter two approaches identified the majority of tools. We describe each of the approaches organized by the primary purpose of the assessment as follows: (i) QI or (ii) research and evaluation, since the issues and selection criteria are different.
|
Quality improvement
To be of value for QI, measures need to be brief, easy and inexpensive to administer and score, and capable of providing useful and ‘real-time’ feedback. QI measures also should allow for comparisons over time and can be benchmarked against data from other clinicians or settings, across populations, or standards of care. Although measurement tracking for QI is complementary in many respects to ongoing data collection for research, a key difference is that measurement tracking for QI has as its primary goal continuous practice improvement (i.e. rapid analysis of data, reporting and feedback).
Below, we describe various approaches to tracking delivery of the 5As for QI. These approaches should be practical for integration into routine care in most settings. The various assessment approaches also can be used in combination to provide a more comprehensive picture.
Checklists are designed to both prompt and record clinician behaviors. Out of the four approaches described, this is perhaps the most efficient, since it is a ‘real-time’ approach to documenting patient–clinician interactions. Checklists can be included in the patient record and provide the clinician with a framework to record follow-up notes. Figure 1 is a checklist that was recently developed for the Health Research & Educational Trust's Pilot Collaborative on Self-Management Support (see www.collaborativeselfmanagement.org). It allows the clinician to ‘walk’ patients through the 5As at a given visit, and to later re-assess progress in meeting collaboratively set goals. A slightly more complex example is shown in Figure 2. This checklist, which embeds 5As issues within counselor responsibilities, was developed specifically for a tobacco use cessation intervention (Hollis et al., 2000). Although the checklist is perhaps the easiest approach to document 5As counseling for QI purposes, clinicians must use it consistently across visits if it is to provide reliable data.
|
|
Patient exit interviews assess patient perspectives on the counseling they received, and evaluate changes in practice patterns related to the 5As. Exit interviews can be conducted on a random sample from the patient population or with a consecutive set of patients at prescribed intervals (e.g. monthly or quarterly) to track changes in practice patterns over time. Figure 3 provides 10 questions that can be used to assess patient perceptions of delivery of the 5As. This question set was used by sites participating in the Pilot Collaborative on Self-Management Support to assess delivery of four of the 5As—Advise, Agree, Assist and Arrange. The Self-Management Pilot Collaborative planning team has also developed a 5-item patient satisfaction survey, which specifically addresses whether the patient thought she received counseling around the 5As, and if so, how satisfied she was with the care received. It can be conducted via patient exit interview or self-administered (Figure 4). Such surveys can be conducted in combination with provider checklists to triangulate clinician-reported delivery of care with patient-reports of care received. This is important, since it is possible for patients to report that they did not receive specific counseling even though the clinician may have reported doing so. From a QI perspective, this could help identify gaps and improvements needed to effectively communicate with patients.
|
|
Medical records can be abstracted to document 5As delivery. Although this is a labor-intensive process, medical chart abstraction provides perhaps one of the most important measures of clinicians' delivery of the 5As. This is because what gets into the medical record is often what other team members see and pay attention to, and such documentation provides a prompt for follow-up on other ‘As’ at future contacts. A limitation of chart audits is that they typically underreport the level of counseling (Stange et al., 1998
; Newell et al., 1999). Chart abstraction can be conducted on a consecutive or random sample of patients and could occur concurrently with other QI efforts. It does require development of coding manuals and definitions to ensure that charts are reviewed in a consistent manner (Boyle and Solberg, 2004). For example, Serdula et al. developed a weight loss counseling tool using the 5As, based on an evidence-based algorithm for the treatment of obesity (Serdula et al., 2003). Criteria used to define delivery of each of the 5As are also available in various publications (Fiore et al., 2000; Glasgow et al., 2003; Glasgow et al., 2004b; Goldstein et al., 2004). DePue et al. utilized a chart audit approach in community health centers to evaluate performance rates of the ‘4As’ (Ask, Advise, Assist, Arrange) for tobacco use cessation counseling (DePue et al., 2002). They audited consecutive records at each health center site, and related quality of care delivery to site level characteristics (e.g. size, location, percentage of providers attending training).
Brief retrospective surveys can help prompt a discussion with patients about goals and creating self-management supports. Self-report measures obtain the patient perspective on receipt of counseling and overcome some of the limitations of provider reports, but are themselves subject to reporting biases and social desirability (Stange et al., 1998). Various research groups have developed brief, face valid questions for patients about the extent to which they have received 5As counseling for specific target behaviors (Sciamanna et al., 2002). Glasgow et al. reported that primary care patients responding to a national survey reported receiving 5As for smoking cessation more often that for other health-related behaviors such as physical activity or healthy eating (Glasgow et al., 2001a). They also reported that the last two A's—assistance and arranging follow-up support—were reported the least often across target behaviors.
With more sophisticated electronic patient information systems, it is possible to integrate clinician-based prompts and feedback with other interactive technologies for self-management support (Glasgow and Bull, 2001; Bodenheimer and Grumbach, 2003; Glasgow et al., 2004a). Patient surveys are most feasible when electronic registries are available that can provide reminders to enhance self-management supports at the point of contact (Glasgow et al., 2001b; McKay et al., 2001) although even manual reminders can be effective in this regard (Marquez, 2001). Glasgow et al. provide an example of a flow diagram that includes prompts for all personnel, including patients, physicians and care managers involved in a typical patient visit (Glasgow et al., 2003). They provide (i) a scenario of what a patient visit might look like that fully integrated self-management support into primary care and (ii) strategies that can be used by various staff members involved with the patient's care before, during and after a visit.
Key characteristics of the QI assessment approaches described above are that they are practical and relatively inexpensive to implement—especially if chart reviews are being done for other purposes (Table 1). Integrating one or more of these into existing QI strategies also can be cost-effective—for example, through a third-party organization like Quality Improvement Organizations (QIOs). Under the direction of Centers for Medicare and Medicaid Services, the QIO program consists of a national network of 53 QIOs responsible for each US state, territory and the District of Columbia. QIOs work with consumers, clinicians, hospitals and other providers to improve care delivery systems, particularly among underserved populations (www.medqic.org).
Finally, the Health Disparities CollaborativesTM offer a number of training manuals and tools available for free download (www.healthdisparities.net). This website offers a widearray of registries, questionnaires, chart audit tools, flow charts and reports for components of the Chronic Care Model (i.e. clinical information systems, community, decision support, delivery system design, organization of health care and self-management support) for asthma, depression and diabetes. Although some of these QI aids are not specifically organized around the 5As, most do involve assessment, assistance and arranging follow-up.
Research and evaluation
Instruments used for research purposes are typically held to higher standards of reliability and validity than QI measures. They also tend to be longer, more expensive and comprehensive, and usually cannot be scored quickly to provide immediate feedback. They are intended to provide a more in-depth understanding of the complexities of care delivery (Table 1).
Direct observation methods
Despite their expense and impracticality in many situations, observational coding systems, based upon live, videotaped or audiotaped clinic–patient interactions are considered the gold standard assessment methods for patient–clinician interactions. To our knowledge, there are no observational coding systems specifically designed for the 5As. There are, however, numerous related systems for coding patient–provider interactions (Roter, 1991). The earliest methods were based on the client-centered counseling approach of Rogers and colleagues and related systems have evolved in the psychotherapy literature to measure constructs such as empathy and percent of talk time for patients versus clinicians (Rogers, 1951; Truax, 1966). More recently, researchers interested in patient-centered interventions have coded patient–staff interactions for key components of motivational interviewing and patient-centered communication style (Amrhein et al., 2003; Baer et al., 2004). Closer to primary care, Stange and colleagues have rated patient–provider interactions for the delivery of preventive services (Stange et al., 2000; Flocke and Stange, 2004; Gilchrist et al., 2004). Results from observational coding systems have contributed greatly to understanding the structure and patterns of patient–clinician communication. These systems can also be of great value for clinician training. Because of their cost, complexity and the lack of real-time feedback, observational coding systems for the 5As are not widely used.
Computer-based assessments of medical records
Electronic chart abstraction tools and coding manuals are available via the Internet. For example, the RAND organization is currently conducting a 4-year study of the effectiveness of chronic illness care collaboratives that utilize the Chronic Care Model (Wagner, 1998; Cretin et al., 2004) across various patient populations, including those with asthma, depression, congestive heart failure and diabetes. They are tracking key processes of care and have developed quality indicators for each disease. Measures related to the 5As associated with the Chronic Care Model are being tracked (i.e. documentation of counseling, goal setting and development of an action plan) (Cretin et al., 2004). These investigators developed a series of Microsoft® Access-based tools to enter the abstracted data (http://www.rand.org/health/ICICE).
Smoking cessation is the area in which the 5As have been used most extensively (Fiore et al., 2000; Hollis et al., 2000; DePue et al., 2002). Kaiser Permanente researchers (Hazlehurst et al., 2005) developed a natural language programming tool to analyze free field text notes in electronic medical records to analyze the extent to which each of the 5As is used for smoking cessation counseling in primary care. They found that while patients were often assessed and advised to quit smoking, seldom were they provided assistance or follow-up support.
Patient surveys
Patient-centered activities such as listening to patients, considering their priorities, developing collaborative goals and eliciting coping suggestions that are congruent with patient values, preferences and social environment are at the heart of the 5As approach. Thus, it is critically important to evaluate patient perceptions of the extent to which they have received such assistance. Although a variety of related instruments have been developed, especially to assess core aspects of primary care and delivery of components of the Chronic Care Model, few survey instruments assess the 5As. Some scales of the ACES (Murray and Safran, 2000) and the Shi et al. assessments of patient perspectives of primary care are related to the first three of the 5As (Shi et al., 2001). Glasgow, Wagner and colleagues reported on the Patient Assessment of Chronic Illness Care (PACIC) (Glasgow et al., 2005a; Glasgow et al., 2005b). This instrument assesses implementation of five activities (patient activation, practice design/decision support, goal setting, problem solving and follow-up) related to the Chronic Care Model that are congruent with the 5As (Glasgow et al., 2005b). Glasgow et al. report that assistance in problem solving and arranging follow-up support (the 4th and 5th As) are reported least often by patients (Glasgow et al., 2005a).
The PACIC has recently been modified to include additional items so that it can be scored specifically for each of the 5As (Glasgow et al., 2005b) (www.collaborativeselfmanagement.org). To our knowledge, this 26-item survey (Table 2) is the only instrument of this level of comprehensiveness scored specifically for the 5As. Although this scale worked well in a pilot collaborative and is an extension of the validated 20-item PACIC, to date the 26-item scale and the 5As scoring has only been formally validated in one study with diabetes patients (Glasgow et al., 2005b). Both the 20- and 26-item versions of the PACIC are in the public domain.
|
|
CONCLUSIONS |
|---|
|
TOPSUMMARYINTRODUCTIONCONCLUSIONSREFERENCES |
|---|
This article reviews instruments that we were able to locate through literature searches and contacts with other colleagues, or developed as part of QI efforts to improve self-management support. Since there are no MeSH terms directly related to the 5As, we cannot be certain that all relevant tools were identified. Also, many of the tools are as yet unpublished. To address this situation, we have provided the tables and figures in this manuscript and posted the tools that are in the public domain on the www.collaborativeselfmanagement.org website.
It is not possible to conclude that any one 5As assessment procedure should be recommended over others. Which instrument is best depends on one's purpose and situational constraints. Different instruments would be selected to definitively identify processes associated with improved outcomes in a multisite clinical trial than for a QI project requiring immediate feedback.
Available literature (Bodenheimer et al., 2002; Bodenheimer and Grumbach, 2003; Glasgow et al., 2003; Hill-Briggs, 2003) suggests that the last two As—assistance in problem solving and arranging follow-up support—are especially important to produce meaningful and lasting behavior change. Unfortunately, they are also the two As that seem to be delivered least often (Glasgow et al., 2001a; Glasgow et al., 2005a). Measures of the 5As should devote special attention to assessment of these components.
We hope that this article will stimulate others to use, validate and refine existing 5As assessment procedures. To our knowledge, at present only the PACIC and the natural language programing methods have supportive validational data. Most of the other tools have face validity and may be more practical, but data are needed on their reliability, validity and especially sensitivity to change. Translation into Spanish and other languages, and validational studies with different ethnic and cultural groups should be a high priority.
|
ACKNOWLEDGEMENTS |
|---|
Preparation of this paper was supported by a grant from the Robert Wood Johnson Foundation, ‘Pilot Collaborative for Self-Management Support’ to Doriane C. Miller.
|
REFERENCES |
|---|
|
TOPSUMMARYINTRODUCTIONCONCLUSIONSREFERENCES |
|---|
Amrhein P. C., Miller W. R., Yahne C. E., Palmer M., Fulcher L. (2003) Client commitment language during motivational interviewing predicts drug use outcomes. Journal of Consulting and Clinical Psychology 71:862–878.[CrossRef][ISI][Medline]
Baer J. S., Rosengren D. B., Dunn C. W., Wells E. A., Ogle R. L., Hartzler B. (2004) An evaluation of workshop training in motivational interviewing for addiction and mental health clinicians. Drug and Alcohol Dependence 73:99–106.[CrossRef][ISI][Medline]
Berwick D. M. (2003) Disseminating innovations in health care. Journal of the American Medical Association 289:1969–1975.
Bodenheimer T. S. and Grumbach K. (2003) Electronic technology: a spark to revolutionize primary care? Journal of the American Medical Association 290:259–264.
Bodenheimer T. S., Lorig K., Holman H., Grumbach K. (2002) Patient self-management of chronic disease in primary care. Journal of the American Medical Association 288:2469–2475.
Boyle R. and Solberg L. I. (2004) Is making smoking status a vital sign sufficient to increase cessation support actions in clinical practice? Annals of Family Medicine 2:22–25.
Cretin S., Shortell S. M., Keeler E. B. (2004) An evaluation of collaborative interventions to improve chronic illness care: framework and study design. Evaluation Review 28:28–51.[Abstract]
DePue J. D., Goldstein M. G., Schilling A., Reiss P., Papandonatos G., Sciamanna C., et al. (2002) Dissemination of the AHCPR clinical practice guideline in community health centers. Tobacco Control 11:239–335.
Fiore M. C., Bailey W. C., Cohen S. J., Dorfman S. F., Goldstein M. G., Gritz E. R., et al. (2000) Treating Tobacco Use and Dependence: Clinical Practice Guideline (U.S. Department of Health and Human Services, Public Health Service, Rockville, MD).
Flocke S. A. and Stange K. C. (2004) Direct observation and patient recall of health behavior advice. Preventive Medicine 38:343–349.[CrossRef][ISI][Medline]
Gilchrist V. J., Stange K. C., Flocke S. A., McCord G., Bourguet C. C. (2004) A comparison of the National Ambulatory Medical Care Survey (NAMCS) measurement approach with direct observation of outpatient visits. Medical Care 42:276–280.[CrossRef][ISI][Medline]
Glasgow R. E. and Bull S. S. (2001) Making a difference with interactive technology: considerations in using and evaluating computerized aids for diabetes self-management education. Diabetes Spectrum 14:99–106.
Glasgow R. E., Eakin E. G., Fisher E. B., Bacak S. J., Brownson R. C. (2001a) Physician advice and support for physical activity: results from a national survey. American Journal of Preventive Medicine 21:189–196.[CrossRef][ISI][Medline]
Glasgow R. E., Hiss R. G., Anderson R. M., Friedman N. M., Hayward R. A., Marrero D. G., et al. (2001b) Report of Health Care Delivery Work Group: behavioral research related to the establishment of a chronic disease model for diabetes care. Diabetes Care 24:124–130.
Glasgow R. E., Funnell M. M., Bonomi A. E., Davis C., Beckham V., Wagner E. H. (2002) Self-management aspects of the improving chronic illness care breakthrough series: implementation with diabetes and heart failure teams. Annals of Behavioral Medicine 24:80–87.[CrossRef][ISI][Medline]
Glasgow R. E., Davis C. L., Funnell M. M., Beck A. (2003) Implementing practical interventions to support chronic illness self-management in health care settings: lessons learned and recommendations. Joint Commission Journal on Quality and Safety 29:563–574.[Medline]
Glasgow R. E., Bull S. S., Piette J. D., Steiner J. (2004a) Interactive behavior change technology: a partial solution to the competing demands of primary care. American Journal of Preventive Medicine 27:80–87.[CrossRef][ISI][Medline]
Glasgow R. E., Goldstein M. G., Ockene J., Pronk N. P. (2004b) Translating what we have learned into practice: principles and hypotheses for addressing multiple behaviors in primary care. American Journal of Preventive Medicine 27:88–101.[CrossRef][ISI][Medline]
Glasgow R. E., Wagner E., Schaefer J., Mahoney L., Reid R., Greene S. (2005a) Development and validation of the Patient Assessment of Chronic Illness Care (PACIC). Medical Care 43:436–444.[CrossRef][ISI][Medline]
Glasgow R. E., Whitesides H., Nelson C. C., King D. K. (2005b) Use of the Patient Assessment of Chronic Illness Care (PACIC) with diabetes patients: relationship to patient characteristics, receipt of care, and self-management. Diabetes Care 28:2655–2661.
Goldstein M. G., Whitlock E. P., DePue J. (2004) Multiple health risk behavior interventions in primary care: summary of research evidence. American Journal of Preventive Medicine 27:2 Suppl, 61–79.[Medline]
Hazlehurst B., Sittig D. F., Stevens V. J., Smith K. S., Hollis J. F., Vogt T. M.,, et al. (2005) Using natural language processing to assess tobacco care quality in the EMR. American Journal of Preventive Medicine 29:434–439.[CrossRef][ISI][Medline]
Hill-Briggs F. (2003) Problem solving in diabetes self-management: a model of chronic illness self-management behavior. Annals of Behavioral Medicine 25:182–193.[CrossRef][ISI][Medline]
Hollis J. F., Bills R., Whitlock E., Stevens V. J. (2000) Implementing tobacco interventions in the real world of managed care. Tobacco Control 9:Suppl I, 18–24.[Medline]
Helping Healthcare Providers Perform According to Standards Marquez L. (2001) Published for the U.S. Agency for International Development (USAID) by the Quality Assurance Project, Bethesda, MD, Operations Research Issue Paper 2(3).
McKay H. G., King D., Eakin E. G., Seeley J. R., Glasgow R. E. (2001) The Diabetes Network Internet-based physical activity intervention: a randomized pilot study. Diabetes Care 24:1328–1334.
Moen R. D., Nolan T. W., Provost L. P. (1999) Quality Improvement Through Planned Experimentation 2nd edition (McGraw Hill, New York City).
Murray A. and Safran D. G. (2000) The Primary Care Assessment Survey: a tool for measuring, monitoring, and improving primary care. In Maruish M. E. (Ed.). Handbook of Psychological Assessment in Primary Care Settings (Lawrence Erlbaum Associates, Mahwah, NJ) pp. 623–651.
National Heart Foundation of Australia and Kinect Australia for the Lifescripts Consortium. (2005) Lifescripts in your division: supporting lifestyle risk factor management in general practice. A guide for Divisions of General Practice. (Commonwealth of Australia, Canberra).
Newell S., Girgis A., Sanson-Fisher R., Savolainen N. (1999) The accuracy of self-reported health behaviors and risk factors relating to cancer and cardiovascular disease in general population: a critical review. American Journal of Preventive Medicine 17:211–229.[CrossRef][ISI][Medline]
Rogers C. (1951) Client-centered Therapy: Its Current Practice, Implications and Theory (Houghton Mifflin, Boston).
Roter D. (1991) Roter Interaction Analysis Coding System (RIAS), Coding Manual (John Hopkins University, Baltimore).
Sciamanna C. N., DePue J. D., Goldstein M. G., Park E. R., Gans K. M., Monroe A. D., et al. (2002) Nutrition counseling in the Promoting Cancer Prevention in Primary Care Study. Preventive Medicine 35:437–446.[CrossRef][ISI][Medline]
Serdula M. K., Khan L. K., Dietz W. H. (2003) Weight loss counseling revisited. Journal of the American Medical Association 289:1747–1750.
Shi L., Starfield B., Xu J. (2001) Validating the adult primary care assessment tool. The Journal of Family Practice 50:161W–175W.
Stange K. C., Zyzanski S. J., Smith T. F., Kelly R., Langa D. M., Flocke S. A., et al. (1998) How valid are medical records and patient questionnaires for physician profiling and health services research? Medical Care 36:851–857.[CrossRef][ISI][Medline]
Stange K. C., Flocke S. A. G. M. A., Kelly R. B., Zyzanski S. J. (2000) Direct observation of rates of preventive service delivery in community family practice. Preventive Medicine 31:167–176 (2 Pt 1).[CrossRef][ISI][Medline]
The Quality Indicator Study Group. (1995) An approach to the evaluation of quality indicators of the outcome of care in hospitalized patients, with a focus on nosocomial infection indicators. Infection Control and Hospital Epidemiology 16:308–316.[ISI][Medline]
The Royal Australian College of General Practitioners. (2004) A population health guide to behavioral risk factors in general practice. Smoking, nutrition, alcohol and physical activity (SNAP) (http://www.racgp.org.au/document.asp?id=?14803).
Truax C. B. (1966) Thereapists' empathy, warmth, genuineness, and patient personality change in group psychotherapy. Journal of Clinical Psychology 22:225–229.[ISI][Medline]
Wagner E. H. (1998) Chronic disease management: what will it take to improve care for chronic illness. Effective Clinical Practice 1:1–4.
CiteULike 
Connotea 
Del.icio.us What's this?
This article has been cited by other articles:
|
|
 |
|
  S. V. Joy Clinical Pearls and Strategies to Optimize Patient Outcomes The Diabetes Educator, May 1, 2008; 34(Supplement_3): 54S - 59S. [Abstract] [Full Text] [PDF]
|
|
| ||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||