Developing Expert-Derived Rating Standards for the Peer Assessment of Lectures.

Acad Med. 2012 Jan 26;

Authors: Newman LR, Brodsky DD, Roberts DH, Pelletier SR, Johansson A, Vollmer CM, Atkins KM, Schwartzstein RM

Abstract
PURPOSE: For peer review of teaching to be credible and reliable, peer raters must be trained to identify and measure teaching behaviors accurately. Peer rater training, therefore, must be based on expert-derived rating standards of teaching performance. The authors sought to establish precise lecture rating standards for use in peer rater training at their school. METHOD: From 2008 to 2010, a panel of experts, who had previously helped to develop an instrument for the peer assessment of lecturing, met to observe, discuss, and rate 40 lectures, using a consensus-building model to determine key behaviors and levels of proficiency for each of the instrument's 11 criteria. During this process, the panelists supplemented the original instrument with precise behavioral descriptors of lecturing. The reliability of the derived rating standards was assessed by having the panelists score six sample lectures independently. RESULTS: Intraclass correlation coefficients of the panelists' ratings of the lectures ranged from 0.75 to 0.96. There was moderate to high positive association between 10 of the 11 instrument's criteria and the overall performance score (r = 0.752-0.886). There were no statistically significant differences among raters in terms of leniency or stringency of scores. CONCLUSIONS: Two relational themes, content and style, were identified within the instrument's variables. Recommendations for developing expert-derived ratings standards include using an interdisciplinary group for observation, discussion, and verbal identification of behaviors; asking members to consider views that contrast with their own; and noting key teaching behaviors for use in future peer rater training.

PMID: 22281550 [PubMed - as supplied by publisher]

Patient safety stories: a project utilizing narratives in resident training.

Acad Med. 2011 Nov;86(11):1473-8

Authors: Cox LM, Logio LS

Abstract
Incident reports have traditionally been the vehicle for identifying, assessing, and responding to quality gaps in hospitals. Yet because of a variety of barriers, residents often fail to participate in this formal process. The authors created a project to engage residents in incident reporting through the use of an online, anonymous narrative format, faculty-facilitated discussion groups, and involvement of patient safety officers in the educational process. During three months, 36 residents submitted a total of 79 stories about patient care that did not "go as intended." The authors reviewed and scored each story for contributing factors and outcomes. The residents met monthly in small groups with trained faculty facilitators to analyze the stories, which were also shared with the patient safety officers. The stories, narratives of both personal involvement and observed events, ranged from near-misses to sentinel events. Key contributing factors included lapses of professionalism, decision errors, communication/information mishaps, transition mix-ups, and workload difficulties. The narrative format proved a feasible tool for collecting significant, previously unrecognized patient safety issues. Internal medicine residents were willing to discuss gaps in care when given the tools and opportunity for anonymous storytelling and blame-free dialogue.

PMID: 21952066 [PubMed - indexed for MEDLINE]

Relating Faults in Diagnostic Reasoning With Diagnostic Errors and Patient Harm.

Acad Med. 2011 Dec 20;

Authors: Zwaan L, Thijs A, Wagner C, van der Wal G, Timmermans DR

Abstract
PURPOSE: The relationship between faults in diagnostic reasoning, diagnostic errors, and patient harm has hardly been studied. This study examined suboptimal cognitive acts (SCAs; i.e., faults in diagnostic reasoning), related them to the occurrence of diagnostic errors and patient harm, and studied the causes. METHOD: Four expert internists reviewed patient records of 247 dyspnea patients, using a specially developed questionnaire to detect SCAs. The patients were treated by 72 physicians between May 2007 and February 2008 in five Dutch hospitals. The findings of the record review were discussed with the treating physicians, and the causes of SCAs were classified using Reason's taxonomy of unsafe acts. Statistical analyses were performed with descriptive statistics and independent t tests to compare groups. Furthermore, a reliability study was conducted to assess the interrater reliability. RESULTS: SCAs occurred in 163 of 247 cases reviewed (66%). In 34 (13.8%) of all cases, a diagnostic error occurred, and in 28 (11.3%) cases, the patient was harmed. Cases with diagnostic errors or patient harm had more SCAs. However, in 10 (4.0%) of the cases, diagnostic errors or patient harm occurred, though there were no SCAs. The causes of SCAs were mostly mistakes (i.e., the planned action was incorrect). CONCLUSIONS: In cases with more SCAs, diagnostic errors and patient harm occurred more often, suggesting that the number of SCAs per case was predictive of the occurrence of these events. The most common causes were mistakes, meaning that physicians did not realize their actions were incorrect.

PMID: 22189886 [PubMed - as supplied by publisher]

Involving Clinical Librarians at the Point of Care: Results of a Controlled Intervention.

Acad Med. 2011 Oct 25;

Authors: Aitken EM, Powelson SE, Reaume RD, Ghali WA

Abstract
PURPOSE: To measure the effect of including a clinical librarian in the health care team on medical residents and clinical clerks. METHOD: In 2009, medical residents and clinical clerks were preassigned to one of two patient care teams (intervention and control). Each team had a month-long rotation on the general medicine teaching unit. The clinical librarian joined the intervention team for morning intake, clinical rounding, or an afternoon patient list review, providing immediate literature searches, formal group instruction, informal bedside teaching, and/or individual mentoring for use of preappraised resources and evidence-based medicine search techniques. Both intervention and control teams completed pre and post surveys comparing their confidence levels and awareness of resources as well as their self-reported use of evidence for making patient care decisions. The nonintervention team was surveyed as the control group. RESULTS: The clinical librarian intervention had a significant positive effect on medical trainees' self-reported ability to independently locate and evaluate evidence resources to support patient care decisions. Notably, 30 of 34 (88%) reported having changed a treatment plan based on skills taught by the clinical librarian, and 27 of 34 (79%) changed a treatment plan based on the librarian's mediated search support. CONCLUSIONS: Clinical librarians on the care team led to positive effects on self-reported provider attitudes, provider information retrieval tendencies, and, notably, clinical decision making. Future research should evaluate economic effects of widespread implementation of on-site clinical librarians.

PMID: 22030761 [PubMed - as supplied by publisher]

Can Eliminating Risk Stratification Improve Medical Residents' Adherence to Venous Thromboembolism Prophylaxis?

Acad Med. 2011 Oct 25;

Authors: Polich AL, Etherton GM, Knezevich JT, Rousek JB, Masek CM, Hallbeck MS

Abstract
PURPOSE: Hospital-acquired venous thromboembolism (VTE) is a common and preventable adverse event that most patients are at risk of developing during their hospital stay. VTE prophylactic anticoagulation (chemoprophylaxis) is the preferred pharmacological assignment for reducing risk of VTE, but it is underused in current practices involving risk stratification (RS) for VTE prevention. The purpose of this study was to determine whether a protocol that eliminates the RS step (non-RS protocol) is more likely to lead residents to evidence-based VTE assignment than the currently used RS protocol. The non-RS protocol follows a methodology that reduces complexity by assuming that the risk of VTE is present and uses contraindications to determine appropriate VTE assignment. METHOD: In 2009, 41 medicine residents at the Nebraska Western Iowa Veterans Affairs clinic participated in an online comparison of two different protocols (RS and non-RS) for assigning chemoprophylaxis for VTE. Six validated, hypothetical patient scenarios were used to compare appropriate (evidence-based) VTE assignments for VTE and completion times for each protocol. RESULTS: Statistical analyses found that the non-RS protocol produced significantly faster (P < .001) scenario completion times and significantly more (P < .001) appropriate VTE assignments than the RS protocol for four of the six patient scenarios. CONCLUSIONS: This study used a new, streamlined protocol (non-RS), which improved VTE assignment and the use of chemoprophylaxis and simplified the process when compared with the use of a traditional RS protocol.

PMID: 22030760 [PubMed - as supplied by publisher]

Restructuring an Inpatient Resident Service to Improve Outcomes for Residents, Students, and Patients.

Acad Med. 2011 Oct 25;

Authors: O?connor AB, Lang VJ, Bordley DR

Abstract
PURPOSE: This study assesses the effects of a resident teaching service restructuring on resident, student, and patient outcomes. METHOD: Interventions included eliminating a "day float" admitting team, converting one-resident:one-intern teams to one-resident:two-intern teams, reducing patient caps from 11 to 7 patients per intern, and increasing pairing between resident teams and attendings. Resident end-of-rotation evaluations and time spent in categorized activities; student end-of-clerkship evaluations, patient logs, and subject exam scores; and hospital-collected patient outcome data were compared before (2007-2008) versus after (2008-2009) the changes. RESULTS: Interns covered fewer patients per day post intervention (9.9 apiece to 6.3 apiece), whereas the total number of patients covered increased (2,501 to 2,916). Enjoyment of the rotation was higher post intervention for interns and senior residents. Residents' time in direct patient care activities and with interns increased post intervention, but residents spent less time with medical students.Students' ratings of several aspects of the clerkship were significantly higher in the postintervention year. Students evaluated more previously unevaluated patients post intervention (32.6% to 45.8%, P < .001), but subject exam scores were unchanged.The median length of stay decreased post intervention (5.0 to 4.0 days, P = .02), and fewer patients required ICU care (11.2% to 7.9%, P < .001). These differences persisted after adjusting for multiple covariates. CONCLUSIONS: An intervention that reduced handoffs and intern patient census and that increased hospitalist pairing was associated with improved resident and student experiences, a favorable impact on patient outcomes, and probable cost savings.

PMID: 22030755 [PubMed - as supplied by publisher]

The Effect of Patient Feedback on Physicians' Consultation Skills: A Systematic Review.

Acad Med. 2011 Sep 26;

Authors: Reinders ME, Ryan BL, Blankenstein AH, van der Horst HE, Stewart MA, van Marwijk HW

Abstract
PURPOSE: The effect of patient feedback interventions as a method of improving physicians' consultation (i.e., communication, interpersonal) skills is equivocal; research is scarce, and methods and rigor vary. The authors conducted this systematic review to analyze the educational effect of feedback from real patients on physicians' consultation skills at the four Kirkpatrick levels. METHOD: The authors searched five databases (PubMed, EMBASE, Cochrane, PsycInfo, ERIC; April 2010). They included empirical studies of all designs (randomized controlled, quasi-experimental, cross-sectional, and qualitative) if the studies concerned physicians in general health care who received formal feedback regarding their consultation skills from real patients. The authors have briefly described aspects of the included studies, analyzed their quality, and examined their results by Kirkpatrick educational effect level. RESULTS: The authors identified 15 studies (10 studies in primary care; 5 in other specialties) in which physicians received feedback in various ways (e.g., aggregated patient reports or educator-mediated coaching sessions), conducted in the United States, the Netherlands, the United Kingdom, Australia, and Canada. All studies that assessed level 1 (valuation), level 2 (learning), and level 3 (intended behavior) demonstrated positive results; however, only four of the seven studies that assessed level 4 (change in actual performance or results) found a beneficial effect. CONCLUSIONS: Some evidence for the effectiveness of using feedback from real patients to improve knowledge and behavior exists; however, before implementing patient feedback into training programs, educators and policy makers should realize that the evidence for effecting actual improvement in physicians' consulting skills is rather limited.

PMID: 21952067 [PubMed - as supplied by publisher]

"Teaching as a Competency": Competencies for Medical Educators.

Acad Med. 2011 Aug 24;

Authors: Srinivasan M, Li ST, Meyers FJ, Pratt DD, Collins JB, Braddock C, Skeff KM, West DC, Henderson M, Hales RE, Hilty DM

Abstract
Most medical faculty receive little or no training about how to be effective teachers, even when they assume major educational leadership roles. To identify the competencies required of an effective teacher in medical education, the authors developed a comprehensive conceptual model.After conducting a literature search, the authors met at a two-day conference (2006) with 16 medical and nonmedical educators from 10 different U.S. and Canadian organizations and developed an initial draft of the "Teaching as a Competency" conceptual model. Conference participants used the physician competencies (from the Accreditation Council for Graduate Medical Education [ACGME]) and the roles (from the Royal College's Canadian Medical Education Directives for Specialists [CanMEDS]) to define critical skills for medical educators. The authors then refined this initial framework through national/regional conference presentations (2007, 2008), an additional literature review, and expert input. Four core values grounded this framework: learner engagement, learner-centeredness, adaptability, and self-reflection.The authors identified six core competencies, based on the ACGME competencies framework: medical (or content) knowledge; learner-centeredness; interpersonal and communication skills; professionalism and role modeling; practice-based reflection; and systems-based practice. They also included four specialized competencies for educators with additional programmatic roles: program design/implementation, evaluation/scholarship, leadership, and mentorship. The authors then cross-referenced the competencies with educator roles, drawing from CanMEDS, to recognize role-specific skills.The authors have explored their framework's strengths, limitations, and applications, which include targeted faculty development, evaluation, and resource allocation. The Teaching as a Competency framework promotes a culture of effective teaching and learning.

PMID: 21869655 [PubMed - as supplied by publisher]

Art Rounds: Teaching Interprofessional Students Visual Thinking Strategies at One School.

Acad Med. 2011 Aug 24;

Authors: Klugman CM, Peel J, Beckmann-Mendez D

Abstract
PURPOSE: The Art Rounds program uses visual thinking strategies (VTS) to teach visual observation skills to medical and nursing students at the University of Texas Health Science Center San Antonio. This study's goal was to evaluate whether students' exposure to VTS would improve their physical observation skills, increase tolerance for ambiguity, and increase interest in learning communication skills. METHOD: In January 2010, 32 students attended three, 90-minute sessions at which they observed and commented on three pieces of art in small groups led by museum educators. Pre and posttest evaluations included Geller and colleagues' version of Budner's Tolerance of Ambiguity Scale, the Communication Skills Attitudes Scale, and free responses to art and patient images. Statistical analyses compared pre and post time looking at images, number of words used to describe images, and number of observations made according to gender and discipline. RESULTS: Students significantly increased the amount of time they spent looking at art and patient images (P = .007), the number of words they used to describe art (P = .002) and patient images (P = .019), and the number of observations made of art (P = .000) and patient images (P = .001). Females increased the time spent observing significantly more than did males (P = .011). Students significantly increased their tolerance for ambiguity (P = .033) and positive views toward health care professional communication skills (P = .001). CONCLUSIONS: The authors speculate that these improved skills may help in patient care and interprofessional team interactions.

PMID: 21869658 [PubMed - as supplied by publisher]

The Incidence and Predictors of Job Burnout in First-Year Internal Medicine Residents: A Five-Institution Study.

Acad Med. 2011 Aug 24;

Authors: Ripp J, Babyatsky M, Fallar R, Bazari H, Bellini L, Kapadia C, Katz JT, Pecker M, Korenstein D

Abstract
PURPOSE: Job burnout is prevalent among U.S. internal medicine (IM) residents and may lead to depression, suboptimal patient care, and medical errors. This study sought to identify factors predicting new burnout to better identify at-risk residents. METHOD: The authors administered surveys to first-year IM residents at five institutions twice between June 2008 and June 2009, linking individual pre- and postresponses. Surveys measured job burnout, sleepiness, personality traits, and other characteristics. Burnout was defined using the most commonly identified definition and another stricter definition. RESULTS: Of 263 eligible residents, 185 (70%) completed both surveys. Among 114 residents who began free of burnout and completed both surveys, 86 (75%) developed burnout, with no differences across institutions. They were significantly more likely to report a disorganized personality style (9 versus 0; 11% versus 0%; P = .019) and less likely to report receiving regular performance feedback (34 versus 13; 63% versus 87%; P = .057). Using a stricter definition, 50% (78/156) of residents developed burnout. They were less likely to plan to pursue subspecialty training (49 versus 63; 78% versus 93%; P = .016) or have a calm personality style (59 versus 70; 77% versus 90%; P = .029). There were no significant associations between burnout incidence and duty hours, clinical rotation, demographics, social supports, loan debt, or psychiatric history. CONCLUSIONS: This study identified a high burnout incidence. The associations observed between burnout incidence and personality style, lack of feedback, and career choice uncertainty may inform interventions to prevent burnout and associated hazards.

PMID: 21869661 [PubMed - as supplied by publisher]

Difficult Issues in Mentoring: Recommendations on Making the "Undiscussable" Discussable.

Acad Med. 2011 Aug 24;

Authors: Bickel J, Rosenthal SL

Abstract
Many mentoring relationships do not reach fruition because the individuals fail to bridge a critical difference. When a difference prevents a learning partnership from achieving its potential, the loss is multidimensional for the individuals and the institution-wasting opportunities for the fostering of current and future talent. Insights into when such impasses are likely to arise may help both mentors and mentees address what feels "undiscussable." The authors offer numerous examples of how differences related to ethnicity, language, gender, and generation may interfere with the development of mentoring relationships. Next, the authors offer recommendations on preparing for and handling difficult conversations, including creating safety, noticing assumptions and emotions, and raising sensitive issues. Virtually all faculty can become more effective at communicating across differences and addressing difficulties that prevent mentoring relationships from achieving their potential. The pay-offs for these efforts are indisputable: increased effect in the limited time available for mentoring, an expanded legacy of positive influence, and enhanced communication and leadership skills. The honing of these relational skills enhances the colleagueship and teamwork on which virtually all research, clinical, and educational enterprises depend. Academic health centers that systematically support mentoring enhance institutional stability, talent development, and leadership capacity.

PMID: 21869662 [PubMed - as supplied by publisher]

Teachers as Learners: The Effect of Bedside Teaching on the Clinical Skills of Clinician-Teachers.

Acad Med. 2011 May 25;

Authors: Wenrich MD, Jackson MB, Ajam KS, Wolfhagen IH, Ramsey PG, Scherpbier AJ

PURPOSE: To assess the impact on full-time faculty's own clinical skills and practices of sustained clinical skills bedside teaching with preclerkship students. METHOD: This was a longitudinal, qualitative study of faculty who provide dedicated ongoing bedside clinical skills teaching for preclerkship medical students. Interviews were conducted during 2003 to 2007 with 31 faculty of the Colleges program at University of Washington School of Medicine. Content analyses of interview transcripts were performed. RESULTS: Teachers perceived a strong positive impact of teaching on their own clinical skills. Six themes were associated with the influence of bedside teaching on teachers' skills and practices. One related to deterrents to change (e.g., reliance on tests/specialists) that narrowed teachers' practice skills prior to starting bedside teaching. Three related to expansion of the process of clinical care resulting from bedside teaching: expanded knowledge and skills, deconstructing the clinical experience (e.g., deepening, broadening, slowing one's practice), and greater self-reflection (e.g., awareness of being a role model). Two were perceived outcomes: improved clinical skills (e.g., physical examination) and more mindful practices (e.g., self-confidence, patient-centered). CONCLUSIONS: Teachers perceived profound positive impact on their clinical skills from teaching preclerkship students at the bedside. Further studies are needed, including comparing teaching preclerkship students with teaching advanced students and residents, to assess whether teaching at other levels has this effect.

PMID: 21617505 [PubMed - as supplied by publisher]

Commentary: is the glass half empty? Code blue training in the modern era.

Acad Med. 2011 Jun;86(6):680-3

Authors: Yang J, Howell MD

Skilled management of cardiopulmonary resuscitation, or responding to a "code blue," is widely considered an important training objective during internal medicine residency. Gaining proficiency in managing a code blue typically depends on event-based experiential learning. In this issue of Academic Medicine, Mickelsen and colleagues report their use of schedule-based stochastic simulation estimates matched with observed code blue data to model the number of annual opportunities a first-year resident has to participate in code blue events. Their data offer compelling evidence that trainees in 2008 had much less opportunity (83% less) to participate in code blue events than did their predecessors in 2002. Mickelsen and coinvestigators speculate that this reduction could be attributable to quality improvement initiatives that may have reduced the total number of code blue situations, as well as to duty hours restrictions that reduced the residents' overall availability to participate. The authors of this commentary discuss the general influence of secular trends on educational needs, and they describe possible strategies to compensate for less "in-the-field" exposure by maximizing the "learning yield per event" and using simulation training methods. Finally, the authors consider the question of whether code blue training remains an appropriate goal for general medicine trainees in the face of evolving trends in health care systems.

PMID: 21613890 [PubMed - in process]

Reduced Resident "Code Blue" Experience in the Era of Quality Improvement: New Challenges in Physician Training.

Acad Med. 2011 Apr 20;

Authors: Mickelsen S, McNeil R, Parikh P, Persoff J

PURPOSE: Emergency resuscitation or "code blue" is a clinical event through which responding medical residents gain experience and proficiency. A retooling of practice has occurred at academic medical centers since the emergence of quality improvement initiatives and resident duty hours limits. The authors investigated how these changes may impact code blue frequency and resident opportunities to gain clinical experience. METHOD: The authors conducted a single-center, retrospective (2002-2009) review of monthly code blue frequency. They compared code blue frequency with corresponding monthly first-year internal medicine resident call schedules (2002-2008 academic years). Using a Monte Carlo simulation they estimated annual code blue experience, and using Poisson regression, they estimated annual trends in resident code blue experience. RESULTS: The authors detected a 41% overall reduction in code blue events between 2002 and 2008; code blue events decreased by 13% annually (P < .001). These trends persisted, even after accounting for hospital census fluctuations: Rates fell from approximately 12 code blue events/1,000 admissions in 2002 to 3.8 events/1,000 in 2008. Overall, the model of code blue frequency and resident call schedules shows a dramatic reduction in the predicted number of code blue experiences, falling from 29 events (empirical 95% CI 18-40) in academic year 2002 to 5 events (CI 1-9) in 2008. CONCLUSIONS: Physicians-in-training at one facility are seeing far fewer code blue events than their predecessors. Whether current numbers of in-hospital code blue events are sufficient to provide adequate experience without supplemental practice for trainees is unclear.

PMID: 21512366 [PubMed - as supplied by publisher]

Bayes' Theorem and the Physical Examination: Probability Assessment and Diagnostic Decision Making.

Acad Med. 2011 Mar 23;

Authors: Herrle SR, Corbett EC, Fagan MJ, Moore CG, Elnicki DM

PURPOSE: To determine how examination findings influence the probability assessment and diagnostic decision making of third- and fourth-year medical students, internal medicine residents, and academic general internists. METHOD: In a 2008 cross-sectional, Web-based survey, participants from three medical schools were asked questions about their training and eight examination scenarios representing four conditions. Participants were given literature-derived preexamination probabilities for each condition and were asked to (1) estimate postexamination probabilities (post-EPs) and (2) select a diagnostic choice (report that condition is present, order more tests, or report that condition is absent). Participants' inverse transformed logit (ITL) mean post-EPs were compared with corresponding literature-derived post-EPs. RESULTS: Of 906 individuals invited to participate, 684 (75%) submitted a completed survey. In two of four scenarios with positive findings, the participants' ITL mean post-EPs were significantly less than corresponding literature-derived post-EP point estimates (P < .001 for each). In three of four scenarios with negative findings, ITL mean post-EPs were significantly greater than corresponding literature-derived post-EP point estimates (P < .001 for each). In the four scenarios with positive findings, 17% to 38% of participants ordered more diagnostic tests when the literature indicated a >85% probability that the condition was present. In the four scenarios with largely negative findings, 70% to 85% chose to order diagnostic tests to further reduce diagnostic uncertainty. CONCLUSIONS: All three groups tended to similarly underestimate the impact of examination findings on condition probability assessment, especially negative findings, and often ordered more tests when probabilities indicated that additional testing was unnecessary.

PMID: 21436660 [PubMed - as supplied by publisher]