No full-text available
To read the full-text of this research,
you can request a copy directly from the authors.
The Effect of Gender Interactions on Students' Physical Examination Ratings in Objective Structured Clinical Examination Stations
Abstract
Previous studies have reached a variety of conclusions regarding the effect of gender on performance in objective structured clinical examinations (OSCEs). Most measured the effect on students' overall OSCE score. The authors of this study evaluated the effect of gender on the scores of specific physical examination OSCE stations, both "gender-sensitive" and "gender-neutral."
In 2008, the authors collected scores for 138 second-year medical students at the University of Calgary who underwent a seven-station OSCE. Two stations--precordial and respiratory exams--were considered gender-sensitive. Multiple linear regression was used to explore the effect of students', standardized patients' (SPs'), and raters' genders on the students' scores.
All 138 students (69 female) completed the OSCE and were included in the analyses. The mean scores (SD) for the two stations involving examination of the chest were higher for female than for male students (83.2% [15.5] versus 78.3% [15.8], respectively, d = 0.3, P = .009). There was a significant interaction between student and SP gender (P = .02). In the stratified analysis, female students were rated significantly higher than male students at stations with female SPs (85.4% [15.5] versus 76.6% [16.5], d = 0.6, P = .004) but not at stations with male SPs (80.2% [15.0] versus 80.0% [15.0], P = 1.0).
These results suggest student and SP genders interact to affect OSCE scores at stations that require examination of the chest. Further investigations are warranted to ensure that the OSCE is an equal experience for all students.
... Previous studies in the field of medical education have identified significant differences in learning between the genders (Bienstock, Martin, Tzou, & Fox, 2002;Carson, Peets, Grant, & McLaughlin, 2010;Minter, Gruppen, Napolitano, & Gauger, 2005;Walker, Kay, McLean, & Haq, 2007). Therefore, to investigate whether participation in the interdisciplinary course led to a differential change in knowledge gain between female and male students, an analysis was carried out between the two genders. ...
... First, the participants had an unequal gender distribution. Previous studies have shown that female and male medical students differ significantly in academic achievement (Bienstock et al., 2002;Carson et al., 2010;Haq, Higham, Morris, & Dacre, 2005) and selfassessment ( Minter et al., 2005). However, by self-analysis, these authors detected no significant difference, which is consistent with our findings reported here. ...
Background: Despite consistent demands, examples of interdisciplinary and evidence-based curricula are rare in dentistry. To address this, the following pilot study describes the conception of an interdisciplinary curriculum on the topic of tooth extraction (TE) vs. tooth preservation (TP). Methods: Based on the German National Learning Objective Catalog on Dentistry, 1408, learning objectives were analyzed, and an interdisciplinary lecture series on TE vs. TP was conceived and conducted. Students' knowledge increase was measured using self-assessment questionnaires on a 6-step Likert scale (from 1 = very good to 6 = insufficient) and by structured evaluation. Results: Overall, the students rated their theoretical knowledge significantly better (p < 0.001) and the pilot course was evaluated "well" to "very well" by the students in terms of didactics, relevance, extent, and involvement. Discussion: The pilot course allowed students to choose between tooth extraction and tooth preservation based on an evidence-based and interdisciplinary assessment.
... We did not attempt to adjust for SPs. Additionally we did not investigate or adjust for gender effects which have been shown to effect grading [29], [30], [31]. Varying gender of SPs might have influenced students' performance at the chest pain station and the acute cough station, where auscultation was within the scope of the demanded skills. ...
... Wir zeigen die Noten einer Station über alle Szenarien als Box-Plots mit Mittelwert, Median, Interquartilenabstand und Ausreißern (siehe Abbildung 1). Die interne Konsistenz der OSCE-Prüfung wurde mittels Cronbach's [29], [30], [31]. Weibliche und männliche SPs wechselten an einigen Stationen, was die Performance der Studierenden an der "Brustschmerz"-Station und der Station mit dem Beratungsanlass "akuter Husten", an denen die Auskultation des Thorax als mögliche klinische Untersuchung in Frage kam, beeinflusst haben kann. ...
Introduction:
The aim of the Objective Structured Clinical Examination
(OSCE) is a standardized and fair assessment of clinical skills. Observing second clinical year medical students during a summative OSCE assessing a General Practice clerkship, we noticed that information exchange with peers led to a progressively faster and overly focused management of simulations. Therefore, we established a Multiple Scenario-OSCE
(MS-OSCE) where all students had to manage the same chief complaint at a station but it’s underlying scenarios being randomly changed during students’ rotation through their parcours. We wanted to ensure they fully explore differential diagnosis instead of managing their task influenced by shared information. We wanted to assess if a MS-OSCE violates the assumption of objectivity and fairness given that students are not tested with the same scenarios.
Methods:
We developed and piloted five OSCE stations (chest pain, abdominal pain, back pain, fatigue and acute cough) with two or three different underlying scenarios each. At each station these scenarios randomly changed from student to student. Performance was assessed with a checklist and global rating. The effect of scenarios and raters on
students’ grades was assessed calculating the intraclass correlation coefficient with a fixed effect two level linear model.
Results:
A total of 169 students and 23 raters participated in the MS-OSCE. The internal consistency over all stations was 0.65 by Cronbach’s alpha. The difference of the mean grades between the scenarios of a given chief complaint ranged from 0.03 to 0.4 on a 1 to 5 grading scale. The effect of scenarios on the variance of the final grades at each station ranged from 4% to 9% and of raters from 20% to 50% when adjusted for students’ skills.
Conclusions:
The effect of different scenarios on the grades was relevant but small compared to the effect of raters on grades. Improving rater training is more important to ensure objectivity and fairness of MS-OSCE than providing the same scenario to all students.
Keywords: medical students, medical education, objective structured clinical examination, rater effects
... A few small studies [12][13][14] have suggested an interaction between student and evaluator gender in the grading of medical students' simulated clinical performance on OSCEs by standardized patients (SPs). One small study of OSCE grading 13 found that male and female medical students fared similarly overall; however, when graded solely by female SPs, women scored significantly higher, yet male and female students were rated the same by male SPs. ...
... One small study of OSCE grading 13 found that male and female medical students fared similarly overall; however, when graded solely by female SPs, women scored significantly higher, yet male and female students were rated the same by male SPs. These findings were replicated in a more recent study of OSCE grading, 14 ...
Purpose:
Clinical performance evaluations are major components of medical school clerkship grades. But are they sufficiently objective? This study aimed to determine whether student and evaluator gender is associated with assessment of overall clinical performance.
Method:
This was a retrospective analysis of 4,272 core clerkship clinical performance evaluations by 829 evaluators of 155 third-year students, within the Alpert Medical School grading database for the 2013-2014 academic year. Overall clinical performance, assessed on a three-point scale (meets expectations, above expectations, exceptional), was extracted from each evaluation, as well as evaluator gender, age, training level, department, student gender and age, and length of observation time. Hierarchical ordinal regression modeling was conducted to account for clustering of evaluations.
Results:
Female students were more likely to receive a better grade than males (adjusted odds ratio [AOR] 1.30, 95% confidence interval [CI] 1.13-1.50), and female evaluators awarded lower grades than males (AOR 0.72, 95% CI 0.55-0.93), adjusting for department, observation time, and student and evaluator age. The interaction between student and evaluator gender was significant (P = .03), with female evaluators assigning higher grades to female students, while male evaluators' grading did not differ by student gender. Students who spent a short time with evaluators were also more likely to get a lower grade.
Conclusions:
A one-year examination of all third-year clerkship clinical performance evaluations at a single institution revealed that male and female evaluators rated male and female students differently, even when accounting for other measured variables.
... Carson et al. recently examined the effect of patient and/or provider gender on gender-sensitive OSCE stations (defined as "a station that might evoke discomfort or embarrassment due to gender differences of the patient, student, or a combination of both") by studying medical student performance at the cardiac and respiratory stations since these examinations include skills performed on the anterior aspect of chest. 7 Female students were rated significantly higher than male students at both the cardiac and respiratory exam stations overall and particularly at the stations with a female SP. 7 Our findings support Carson's conclusion that the interaction of provider gender and patient gender has an important effect on performance of gender-sensitive examinations including the cardiac exam. ...
... 7 Female students were rated significantly higher than male students at both the cardiac and respiratory exam stations overall and particularly at the stations with a female SP. 7 Our findings support Carson's conclusion that the interaction of provider gender and patient gender has an important effect on performance of gender-sensitive examinations including the cardiac exam. Since our study evaluated the performance of individual cardiac examination skills rather than overall performance at a cardiac OSCE station, we are also able to more specifically identify "gender-sensitive" maneuvers within the cardiac exam as being those that are closest to the breast, including auscultation of the mitral valve region and palpation of the apical impulse. ...
Background:
Physical examination remains an important part of the initial evaluation of patients presenting with chest pain but little is known about the effect of patient gender on physician performance of the cardiovascular exam.
Objective:
To determine if resident physicians are less likely to perform five key components of the cardiovascular exam on female versus male standardized patients (SPs) presenting with acute chest pain.
Design:
Videotape review of SP encounters during Objective Structured Clinical Examinations (OSCEs) administered by the Emory University Internal Medicine Residency Program in 2006 and 2007. Encounters were reviewed to assess residents' performance of five cardiac exam skills: auscultation of the aortic, pulmonic, tricuspid, and mitral valve areas and palpation for the apical impulse.
Participants:
One hundred forty-nine incoming residents.
Main measures:
Residents' performance for each skill was classified as correct, incorrect, or unknown.
Key results:
One hundred ten of 149 (74 %) of encounters were available for review. Residents were less likely to correctly perform each of the five skills on female versus male SPs. This difference was statistically significant for auscultation of the tricuspid (p = 0.004, RR = 0.62, 95 % CI 0.46-0.83) and mitral (p = 0.007, RR = 0.58, 95 % CI = 0.41-0.83) valve regions and palpation for the apical impulse (p < 0.001, RR = 0.27, 95 % CI = 0.16-0.47). Male residents were less likely than female residents to correctly perform each maneuver on female versus male SPs. The interaction of SP gender and resident gender was statistically significant for auscultation of the mitral valve region (p = 0.006) and palpation for the apical impulse (p = 0.01).
Conclusions:
We observed significant differences in the performance of key elements of the cardiac exam for female versus male SPs presenting with chest pain. This observation represents a previously unidentified but potentially important source of gender bias in the evaluation of patients presenting with cardiovascular complaints.
... Prior studies have found that gender can impact the performance of an examination. For example, a study found that male medical students performed worse than female peers in cardiac and pulmonary examinations on female patients [26]. Another study found that both male and female doctors, but especially male doctors, were less likely to correctly perform the cardiovascular exam on female patients, particularly during auscultation in the tricuspid and mitral areas, which are close to the breast [27]. ...
For patients who are often embarrassed and uncomfortable when exposing their breasts and having them touched by physicians of different genders during auscultation, we are developing a robotic system that performs auscultation over clothing. As the technical issue, the sound obtained through the clothing is often attenuated. This study aims to investigate clothing-induced acoustic attenuation and develop a suppression method for it. Because the attenuation is due to the loss of energy as sound propagates through a medium with viscosity, we hypothesized that the attenuation is improved by compressing clothing and shortening the sound propagation distance. Then, the amplitude spectrum of the heart sound was obtained over clothes of different thicknesses and materials in a phantom study and human trial at varying contact forces with a developed passive-actuated end-effector. Our results demonstrate the feasibility of the attenuation suppression method by applying an optimum contact force, which varied according to the clothing condition. In the phantom experiments, the attenuation rate was improved maximumly by 48% when applying the optimal contact force (1 N). In human trials, the attenuation rate was under the acceptable attenuation (40%) when applying the optimal contact force in all combinations in each subject. The proposed method promises the potential of robotic auscultation toward eliminating gender bias.
... The skill to elicit the correct physical findings of a patient is important in combination with history taking to start the clinical reasoning process. Many universities already assess physical examination skills in OSCEs [46,47]. Hence, our competence-based assessment does not necessarily have to test this skill. ...
Background:
Assessing competence of advanced undergraduate medical students based on performance in the clinical context is the ultimate, yet challenging goal for medical educators to provide constructive alignment between undergraduate medical training and professional work of physicians. Therefore, we designed and validated a performance-based 360-degree assessment for competences of advanced undergraduate medical students.
Methods:
This study was conducted in three steps: 1) Ten facets of competence considered to be most important for beginning residents were determined by a ranking study with 102 internists and 100 surgeons. 2) Based on these facets of competence we developed a 360-degree assessment simulating a first day of residency. Advanced undergraduate medical students (year 5 and 6) participated in the physician's role. Additionally knowledge was assessed by a multiple-choice test. The assessment was performed twice (t1 and t2) and included three phases: a consultation hour, a patient management phase, and a patient handover. Sixty-seven (t1) and eighty-nine (t2) undergraduate medical students participated. 3) The participants completed the Group Assessment of Performance (GAP)-test for flight school applicants to assess medical students' facets of competence in a non-medical context for validation purposes. We aimed to provide a validity argument for our newly designed assessment based on Messick's six aspects of validation: (1) content validity, (2) substantive/cognitive validity, (3) structural validity, (4) generalizability, (5) external validity, and (6) consequential validity.
Results:
Our assessment proved to be well operationalised to enable undergraduate medical students to show their competences in performance on the higher levels of Bloom's taxonomy. Its generalisability was underscored by its authenticity in respect of workplace reality and its underlying facets of competence relevant for beginning residents. The moderate concordance with facets of competence of the validated GAP-test provides arguments of convergent validity for our assessment. Since five aspects of Messick's validation approach could be defended, our competence-based 360-degree assessment format shows good arguments for its validity.
Conclusion:
According to these validation arguments, our assessment instrument seems to be a good option to assess competence in advanced undergraduate medical students in a summative or formative way. Developments towards assessment of postgraduate medical trainees should be explored.
... 27e29 It was reported that students' performance in chest examinations were influenced by the genders of students and standard patients. 28 In a university hospital in China, female students had a lower average error frequency than male students in cardiac examinations. 29 Our analysis revealed that female students had lower PE confidence scores than male students. ...
Background:
Physical examination (PE) is a basic diagnostic tool in clinical settings. It is important to enhance medical students' skills and confidence in PE.
Subjects and methods:
Our medical students begin learning PE in the fourth medical year (M4). They undergo hands-on clinical training in the fifth medical year (M5). To improve the teaching of PE, we implemented an advanced PE training course (APETC) for M5 students. In APETC, PE skills for chest (CH), cardiovascular system (CV), and abdomen (GI) domains were demonstrated by senior specialists. Under tutors' supervision, students performed PE on selected patients with positive signs. To evaluate the effect of this training course, we designed a checklist to evaluate students' confidence level in PE before and after APETC. Average confidence scores of PE in CH, CV, and GI domains among different years and genders were compared using ANOVA tests.
Results:
M5 students' confidence in PE for individual items on the CH, CV, and GI checklists increased significantly after the APETC, in both 2014 and 2015 (all p < 0.0001).The average post-course confidence scores of all items in CH, CV, and GI domains increased significantly over average pre-course confidence scores. The average post-course confidence score in PE was higher in the year 2015 than that in the year 2014. Male students tended to have higher confidence scores in PE than female students.
Conclusion:
APETC enhanced medical students' confidence in PE. Long-term effects of the training course and the impact of students' gender on learning outcomes remain to be investigated.
... This study's result contradict with the study conducted by Chan, et al (2014) who found that both genders performed similarly in different aspects (academic, clinical, psychological, nursing profession identity, and health concept). Other opposing study done by Carson et al (2010) who analyzed scores from 138 medical students in their second year of study, found that female students performed better than male students. It can be assumed that different demographic areas and students' characteristics background in Indonesia may cause this discrepancies in the research results. ...
... Another limitation to this study is the unknown distribution of gender in the MS and DS groups between 2008 and 2015, as gender was not electronically recorded in the OSCE analysis, and it has been shown that gender can affect performance outcomes in clinical examinations such as an OSCE [22][23][24]. An internal analysis of total female and male student numbers between 2011 and 2015 (earlier statistics were not available), however, revealed a female-to-male ratio of 2.15 for MS and 2.09 for DS, indicating a similar gender distribution in MS and DS for the analyzed OSCE scenarios. ...
Purpose
To compare the performance in oral, craniomaxillofacial, and facial plastic surgery (CMF)-specific surgical skills between medical students (MS) and dental students (DS) and hence adjust the current CMF training to student-specific needs. The investigators hypothesized that there would be no performance differences between MS and DS.
Methods
The investigators implemented a comparative retrospective item-based analysis of student performance in a CMF-specific objective structured clinical examination (OSCE) from 2008 to 2015. The sample was composed of 1010 MS and 225 DS who completed a standardized CMF training and OSCE. Three OSCE scenarios [management mandible fracture (MMF), management zygomatic fracture (MZF), and structured facial examination (SFE)] were included in the study because learning objectives were equal. Descriptive and bivariate statistics were computed and the p value was set at 0.05.
Results
In all of the analyzed OSCE scenarios, DS significantly outperformed MS (MMF p<0.001; MZF p=0.013; SFE p<0.001). DS especially appeared to be better in the correct interpretation of radiological findings (five of seven items, MMF) and the correct allocation of anatomical structures (four of five items, MZF) as well as the symptom-oriented examination of the eye (three of three items, SFE).
Discussion and conclusion
DS overall seem to be perform better in typical CMF skills. The reasons for this performance gap could be a more profound knowledge of the facial anatomy as well as a higher awareness for CMF as a related specialty to dentistry. CMF should be included in medical curricula in a larger scale, and possible career paths should be highlighted to MS and DS to raise attraction for the specialty. Further studies should focus on the implementation of modern teaching methods in CMF education.
... However, some studies have found that a student's gender can impact how an observer rates a student's clinical performance (Berg et al., 2014;Carson, Peets, Grant, & McLaughlin, 2010). Our group comparisons showed that there were no differences in the way students were rated based upon their gender, suggesting that the rating scales and raters are not affected by gender. ...
The goals of this quasi-experimental pilot study were to test an assessment tool designed to evaluate students' teamwork skills, and to assess the effectiveness of an interprofessional education (IPE) course. Participants were health professional students (physical therapy, pharmacy, dental and osteopathic medicine) 24 of whom were second-year students who had previously taken part in an IPE course (experimental group), and 22 of whom were third years that had not (control group). Students interacted with a standardized patient and her son during an asynchronous Team Objective Structured Clinical Exam (TOSCE), after which they were scored on their teamwork skills using newly designed teamwork rating scales. Cronbach Alpha calculations suggest that the rating scales are reliable when rater scores are aggregated (0.81). Pearson coefficient calculations determined that teamwork scores of live raters and video raters were significantly correlated (p < 0.0001) suggesting good consistency across these raters, and the experimental group performed significantly better (p = 0.0003) than the control group suggesting that the IPE curriculum is successfully equipping students with teamwork skills. The results of this study contribute to the much needed IPE assessment literature, and suggest that teamwork skills can be taught and effectively assessed using this new rating scale.
... In addition, e-Learning methodologies, multi-source evaluation and objective structured clinical examination (OSCE) stations are both well-established in the reformed curriculum and house staff training at Charité and will be explored further in the context of teaching communication skills [33][34][35]. OSCE stations have become one of the key elements in teaching and testing practical and communication skills as well as timely medical decision-making and have also been proven useful in continued medical education of house staff and faculty [36][37][38][39][40][41]. ...
Not only the general public, but also those studying to become health professionals, are struggling to keep up with a growing body of evidence and increasingly complex information about the many different types of vaccines available to date. At the same time, a number of increasingly complex subjects of study are competing for their attention during undergraduate and graduate education. In many medical school curricula in German-speaking countries, the subject of vaccines has been entirely omitted, or is regarded a minor subtopic. During the studies, most medical school curricula in German-speaking countries do not offer obligatory courses and/ or hands-on training vaccinology in vaccination. In Germany, private pediatricians administer the majority of immunizations. Even during postgraduate training programs in pediatrics, which are largely hospital-based, vaccinations are rarely a topic, and vaccinology remains a “hobby” and a “field without lobby” lacking specific certification requirements. Studies of acceptance of vaccines among health professionals and medical students have shown that many may still have their own doubts and uncertainties about vaccines revealing a number of unanswered questions during their studies and postgraduate training.
... Par exemple, des étudiants ayant participé à une station sur la gynécologie-obstétrique tendent à rechercher des notions en lien avec cette discipline quand ils participent ensuite à une station sur la psychiatrie 21 . Les interactions entre le genre de l'étudiant et celui du patient standardisé induisent également un biais, en particulier pour les stations nécessitant un examen de la poitrine : les femmes ont des résultats significativement supérieurs à ceux des hommes avec des patientes standardisées (mais il n'y a pas de différence avec des patients de sexe masculin) 22 . Enfin, à titre expérimental, l'utilisation d'un patient virtuel dans une station ECOS, composé d'une simulation sur ordinateur et d'un assistant humain, a été évaluée. ...
Current normative or formative assessments of medical students tend to focus more
on skills rather than knowledge. The script concordance test has good psychometric
qualities to assess reasoning skills required to resolve complex problems in situations
of uncertainty. Objective structured clinical examinations assess clinical competence
in realistic situations (clinical situation, standardized and simulated patients,
neutral observers, observation check lists) however their organization is complex
and costly. The introduction of human factors (panel of experts, simulated and
standardized patients and observers) reduces the objectivity and reliability of both
these tests. The generation Y (births between 1980 and 1992) has grown up in
the digital age with widespread access to technology and as such expect new and
innovative teaching aids. Serious games, developed in the health education field,
could be used as a student assessment tool. This requires the convergence between
medical science and information technology. Moreover, its implementation should
meet the needs of the educational spiral (teaching needs, method, objectives and
assessment). The move towards digital assessment could open the path towards a
new way of evaluating skills in medicine.
... Virtual medical visits have shown that female patients were more satisfied with female physicians' more caring communication style [28]. Interestingly, in another study female students were also rated significantly higher than male students at OSCE stations with female standardized patients [29]. As discussed by the authors, this might also reflect more patient-centred interactions and communication patterns of female students [30] or different perception of empathy by female patients like we observed in our study. ...
Background
History taking and empathetic communication are two important aspects in successful physician-patient interaction. Gathering important information from the patient’s medical history is needed for effective clinical decision making while empathy is relevant for patient satisfaction. We wanted to investigate whether medical students near graduation are able to combine both skills as required in daily medical practice.
Methods
Thirty near graduates from Hamburg Medical School participated in an assessment for clinical competences including a consultation hour with five standardized patients. Each patient interview was videotaped and standardized patients rated participants with the CARE questionnaire for consultation and relational empathy. All videotaped interviews were rated with a checklist based on the number of important medical aspects for each case. Data were analysed with the linear mixed model to correct for random effects. Regression analysis was performed to look for correlations between the number of questions asked by a participant and their respective empathy rating.
Results
Of the 123 aspects that could have been gathered in total, students only requested 56.4% (95% CI 53.5-59.3). While no difference between male and female participants was found, a significant difference (p < .001) was observed between the two parts of the checklist with 61.1% (95% CI 57.9-64.3) of aspects asked for in part 1 (patient’s symptoms) versus 52.0 (95 47.4-56.7) in part 2 (further history). All female standardized patients combined rated female participants (mean score 14.2, 95% CI 12.3-16.3) to be significantly (p < .01) more empathetic than male participants (mean score 19.2, 95% CI 16.3-22.6). Regression analysis revealed no correlation between the number of medical aspects gathered by a participant and his or her respective empathy score given by the standardized patient in the CARE questionnaire.
Conclusion
Gathering sufficient medical data from a patient’s history and empathetic communication are two completely separate sides of the coin of history taking. While both skills have to be acquired during medical school training with particular focus on their respective learning objectives, medical students need to be provided with additional learning and feedback opportunities where they can be observed exercising both skills combined as required in physicians’ daily practice.
Patient‐dentist discourse is a core nonoperational competency in dental education. The skills of querying patients and responding to questions are noncognitive attributes, and their evaluation by the standardized patient objective structured clinical examination (OSCE) is particularly necessary. However, it is not clear whether students' test anxiety affects these attributes. This study aims to examine the relationship between dental students' state‐trait anxiety, noncognitive performance, and examination results during their first OSCE. A single dental school cohort (n = 226) of 5 year students attending their first clinical examination from 2014 to 2017 was studied. Participants completed the Chinese Mandarin Version State‐Trait Anxiety Inventory Y form before taking the OSCE. The difference between state and trait anxiety levels was compared by paired t test. Gender differences and the effect of age group in these anxiety levels were analyzed using multivariate analysis of variance. Moreover, gender, age group, state anxiety, and trait anxiety scores were compared with the OSCE items of noncognitive performance using a chi‐square test. Students showed significantly higher state anxiety than trait anxiety levels; moreover, women showed significantly higher state anxiety than men. Furthermore, gender, age group, state anxiety, and trait anxiety had no association with the noncognitive performance examination results. Most participants showed moderate state and trait anxiety levels during their first OSCE. Further, the state‐trait anxiety had no significant effect on their noncognitive performance. However, 26.5% of participants did not pass the examination; therefore, dental educators should increase communication skill training courses during clerkship training to improve students' noncognitive attributes.
Background:
For the past 30 years, there has been a growing emphasis on communication and self-evaluation skills training in graduate medical education. This is reflected in the Next Accreditation System. The Objective Structured Clinical Examination (OSCE) is widely used in graduate medical education for assessing dimensions of interpersonal communication and counseling skills. The OSCEs may be developed to target challenging clinical scenarios difficult to capture in clinical practice and can be used as a medium for resident self-evaluation.
Objectives:
The aims of the study were to evaluate residents' interpersonal, communication, and counseling skills using Kalamazoo Essential Elements Communication Checklist in 4 clinically challenging scenarios and to compare standardized patient (SP) evaluations to residents' self-evaluation by category of medical school.
Methods:
South East Michigan Center for Medical Education is a consortium of teaching hospitals. Member residents participate in 4 OSCEs as part of their postgraduate 1 curriculum. The OSCEs were developed to evaluate clinically relevant but difficult to capture scenarios including: (a) error disclosure/counseling an angry patient; (b) delivering bad news/end of life; (c) domestic violence; and (d) counseling a patient with colon cancer requesting alternative treatments. At the conclusion of each OSCE, SPs evaluated and residents self-evaluated their performance. Once evaluations were completed, SPs provided residents with feedback.
Results:
Six member institutions and 344 residents participated during the 2014, 2015, and 2016 academic years. There were more international medical graduates (59%) than graduates of Liaison Committee for Medical Education-accredited medical schools. There were more males (62.2%) than females. Standardized patients rated residents higher than residents rated themselves in 2014 (<0.001), but not in 2015 or 2016. When combining all years and all residents, there was no correlation of SP and resident scores. Standardized patients rated female residents higher than female residents rated themselves (P < 0.0001). Male residents scored themselves similarly to the SPs, but male residents rated themselves higher than female residents rated themselves (P < 0.001). Standardized patient scores for male and female residents were not significantly different.
Conclusions:
Targeted OSCEs provide an objective format to evaluate residents in challenging clinical scenarios. Resident self-evaluations did not correlate with SPs. In addition, female residents rated themselves lower than male residents and lower than SPs. There is need to develop interventions and curricula to improve resident's self-evaluation skills and in particular address lower self-evaluation by female trainees.
Physicians are key in identifying and addressing adolescent high-risk behaviors. We evaluated the influence of patient and student gender on medical students’ assessment of adolescent behaviors. Performance on an adolescent Objective Structured Clinical Exam (OSCE) was compared between male and female students and the four student-patient gender pairs. From 2009 to 2013, 396 students completed the OSCE. Overall scores were significantly higher for females. All students scored poorly on the question addressing sexual behavior, but female students scored significantly higher than males. Students are weak in screening adolescent sexual behaviors and gender impacts screening practices. Further research evaluating this impact is needed.
Objective:
Empathy is a crucial skill for medical students that can be difficult to evaluate. We examined if self-reported empathy in medical students was associated with clinical competence.
Methods:
This study combined cross-sectional data from four consecutive years of medical students (N=590) from the Boston University School of Medicine. We used regression analysis to evaluate if self-reported empathy (Jefferson Scale of Physician Empathy (JSPE)) predicted scores in clinical clerkships, United States Medical Licensing Examinations, and OBJECTIVE: Structured Clinical Examinations (OSCEs). We separately analyzed overall and OSCE communication scores based on interpersonal skills reported by standardized patients. We controlled for age, gender, debt, and specialty affinity.
Results:
JSPE scores of medical students were positively associated with OSCE communication scores, and remained significant when controlling for demographics. We found that JSPE score was also predictive of overall OSCE scores, but this relationship was confounded by gender and age. JSPE scores were associated with performance in the Pediatrics clerkship, but not other clerkships or standardized tests.
Conclusion:
JSPE scores were positively associated with OSCE communication scores in medical students.
Practice implications:
This study supports that self-reported empathy may predict OSCE performance, but further research is needed to examine differences by gender and age.
This chapter will focus on the use of standardized and simulated patients in pediatric healthcare education. The chapter will begin with a section on common terms and definitions followed by a review of the Association of Standardized Patient Educators (ASPE) standards of best practice. General information with regard to recruiting and training will be provided, and the differences in training for formative and summative assessments will be discussed. The use of children and adolescents as standardized and simulated patients will also be explored. Current trends in pediatric simulation, including the use of standardized and simulated patients to improve communication, the development of interpersonal skills, and the delivery of difficult news, will be presented along with the use of hybrid simulations. The chapter will close with a discussion on the advantages and challenges of using standardized and simulated patients in pediatrics.
Purpose:
Previous studies have shown that the standardized patient's (SP) gender may affect student performance in an Objective Structured Clinical Examination (OSCE). The aim of this study is to investigate the influence of the SPs' body habitus on students' performance in an OSCE counseling station.
Methods:
Four equally trained female SPs, with either a normal or an obese BMI participated in an OSCE counseling station for cardiovascular risk factors. Ninety-two, second year medical students were randomly assigned to one of the SPs. Station scores were compared and student behavior and opinion regarding the influence of their SP's body habitus on their performance was assessed.
Results:
There was no difference in mean exam scores for students interacting with SPs with a normal BMI versus increased BMI (14.9 ± 2.2 versus 14.01 ± 2.2/20 respectively, p = 0.06). Additionally, almost all students gave advice about healthy diets (93.5% versus 95.7%) with no specificity regarding the BMI of the SP.
Conclusions:
The body habitus of the SP did not significantly affect students' performance in an undergraduate OSCE about cardiovascular risk factors, suggesting that students at that level may primarily focus on gaining points the diagnostic checklist without considering SPs as real patients.
Staged 2 different videotaped interviews with the same individual—a college instructor who spoke English with a European accent. In one of the interviews the instructor was warm and friendly, in the other, cold and distant. 118 undergraduates were asked to evaluate the instructor. Ss who saw the warm instructor rated his appearance, mannerisms, and accent as appealing, whereas those who saw the cold instructor rated these attributes as irritating. Results indicate that global evaluations of a person can induce altered evaluations of the person's attributes, even when there is sufficient information to allow for independent assessments of them. Furthermore, Ss were unaware of this influence of global evaluations on ratings of attributes. In fact, Ss who saw the cold instructor actually believed that the direction of influence was opposite to the true direction. They reported that their dislike of the instructor had no effect on their rating of his attributes but that their dislike of his attributes had lowered their global evaluations of him. (PsycINFO Database Record (c) 2012 APA, all rights reserved)
To review the literature on gender differences and issues of self-confidence in medical students and to present original research on observers' perceptions of medical student confidence.
One hundred forty-one 3rd year medical students at Indiana University School of Medicine were videotaped during their objective structured clinical examination (OSCE). Trained coders rated how confident the student appeared and coded a variety of nonverbal behaviors at the beginning, middle, and end of the interaction. Analysis focused on gender differences in coders' ratings of perceived confidence.
Female medical students were viewed as significantly less confident than male medical students (F(1,133)=4.45, p<0.05), especially at the beginning of the interaction.
Past research indicates that despite performing equally to their male peers, female medical students consistently report decreased self-confidence and increased anxiety, particularly over issues related to their competence. In a standardized patient interaction examination situation, female medical students also appeared significantly less confident than male medical students to independent observers.
Medical educators should focus on issues of female students' confidence, increasing faculty sensitivity, and publicly recognizing and discussing perceptions of confidence.
The purpose of this study was to explore possible performance differences in interpersonal skills (IPS) ratings as a function of candidate and standardized patient (SP) gender.
The IPS scores and SP characteristics for 79,999 patient encounters were studied. This included 18,325 (20.36%) female candidate to female SP, 26,872 (29.86%) male candidate to female SP, 18,281 (20.31%) female candidate to male SP, and 16,521 (29.47%) male candidate to male SP interactions.
The analysis did not reveal a significant candidate gender by SP gender effect. There were no meaningful differences in IPS scores as a function of SP or candidate gender.
The non-significant interaction between SP gender and candidate gender provides some evidence that male and female candidates are being assessed equivalently by male and female SPs. This result, combined with the extremely weak relationship between gender (candidate or SP) and IPS ratings, provides additional support for the fairness and defensibility of the IPS measures.
Being able to predict medical school performance is essential to help ensure the supply of quality physicians. The purpose of our study was to examine the influence of gender and age on academic performance (AP) and on academic difficulty (AD). The study involved all matriculants of 3 classes at one medical school. Independent variables included gender, age (categorized into younger and older than 23 years) and the gender by age interaction. Dependent variables included an AP scale score, a clinically based performance examination and AD. The Wilson AP Scale score was developed to assess both excellent and poor performance. The Wilson AP Scale included first-, second-, and third-year medical school grade-point-averages, USMLE Step 1score and USMLE Step 2 score. Older women as a group had the highest mean Wilson AP Scale score. Women performed better than men on the clinically based performance examinations. Younger men were least likely to have AD and younger women were most likely to have AD. Five of 123 younger men versus 13/66 older men had AD. Also, 15/63 younger women had AD versus 2/27 older women. A significant gender by age interaction was present in predicting the Wilson AP Scale score (p = 0.009) and AD (p = 0.002). Older women performed better than both older men and younger women in 3 classes of medical students at one medical school. A significant gender by age interaction was predictive of AP and AD. These findings may have implications on admission decisions.
To reduce inter-rater variability in evaluations and the demand on physician time, standardized patients (SP) are being used as examiners in OSCEs. There is concern that SP have insufficient training to provide valid evaluation of student competence and/or provide feedback on clinical skills. It is also unknown if SP ratings predict student competence in other areas. The objectives of this study were: to examine student attitudes towards SP examiners; to compare SP and physician evaluations of competence; and to compare predictive validity of these scores, using performance on the multiple choice questions examination (MCQE) as the outcome variable.
This was a cross-sectional study of third-year medical students undergoing an OSCE during the Internal Medicine clerkship rotation. Fifty-two students rotated through 8 stations (6 physician, 2 SP examiners). Statistical tests used were Pearson's correlation coefficient, two-sample t-test, effect size calculation, and multiple linear regression.
Most students reported that SP stations were less stressful, that SP were as good as physicians in giving feedback, and that SP were sufficiently trained to judge clinical skills. SP scored students higher than physicians (mean 90.4% +/- 8.9 vs. 82.2% +/- 3.7, d = 1.5, p < 0.001) and there was a weak correlation between the SP and physician scores (coefficient 0.4, p = 0.003). Physician scores were predictive of summative MCQE scores (regression coefficient = 0.88 [0.15, 1.61], P = 0.019) but there was no relationship between SP scores and summative MCQE scores (regression coefficient = -0.23, P = 0.133).
These results suggest that SP examiners are acceptable to medical students, SP rate students higher than physicians and, unlike physician scores, SP scores are not related to other measures of competence.
Objective
This research was performed to study the attitudes that medical students hold concerning their relationships with patients, and whether such attitudes are gender-related, affect career plans, and influence their evaluation of psycho-social and biomedical issues.
Methods
One hundred fifty-three first year students at the Boston University School of Medicine completed the Patient-Practitioner Orientation Scale (PPOS), a scale that differentiates between a patient-centered vs. doctor-centered orientation toward medical practice, indicated their interest in community and primary care practice, and rank ordered psycho-social and biomedical clinical issues in terms of their perceived relative importance.
Results
The data revealed that female medical students were more patient-centered, and that (across sexes) patient centeredness was positively associated with an interest in community and primary care practice and the ranking of psycho-social issues.
Conclusions
These findings indicate that differences in the practice attitudes of males and females exist very early on in medical training, and that these differences are associated with anticipated career choices. They also suggest that the PPOS may prove useful in measuring the attitudes of practicing physicians toward their clinical roles and might predict physicians' behavioral strategies and patient medical outcomes.
The objective structured clinical examination (OSCE) requires the use of standardised patients (SPs). Recruitment of SPs can be challenging and factors assumed to be neutral may vary between SPs. On stations that are considered gender-neutral, either male or female SPs may be used. This may lead to an increase in measurement error. Prior studies on SP gender have often confounded gender with case.
The objective of this study was to assess whether a variation in SP gender on the same case resulted in a systematic difference in student scores.
At the University of Ottawa, 140 Year 3 medical students participated in a 10-station OSCE. Two physical examination stations were selected for study because they were perceived to be 'gender-neutral'. One station involved the physical examination of the back and the other of the lymphatic system. On each of the study stations, male and female SPs were randomly allocated.
There was no difference in mean scores on the back examination station for students with female (6.96/10.00) versus male (7.04/10.00) SPs (P = 0.713). However, scores on the lymphatic system examination station showed a significant difference, favouring students with female (8.30/10.00) versus male (7.41/10.00) SPs (P < 0.001). Results were not dependent on student gender.
The gender of the SP may significantly affect student performance in an undergraduate OSCE in a manner that appears to be unrelated to student gender. It would be prudent to use the same SP gender for the same case, even on seemingly gender-neutral stations.
The Medical Council of Canada (MCC) administers a qualifying examination for the issuance of a license to practice medicine. To date, this examination does not test the clinical skills of history taking, physical examination, and communication. The MCC is implementing an objective structured clinical examination (OSCE) to test these skills in October 1992. A pilot examination was developed to test the feasibility, reliability, and validity of running a multisite, two-form, four-hour, 20-station OSCE for national licensure. In February 1991, 240 volunteer first- and second-year residents were tested at four sites. The candidates were randomly assigned to one of two forms of the test and one of two sites for two of the four sites. Generalizability analysis revealed that the variance due to form was 0.0 and that due to site was .16 compared with a total variance of 280.86. The reliabilities (inter-station) were .56 and .60 for the two forms. Station total-test score correlations, used to measure station validity, were significant for 38 of the 40 stations used (range .14-.60). The results of the OSCE correlated moderately with the MCC qualifying examination; these correlations were .32 and .35 for the two test forms. Content validity was assessed by postexamination questionnaires given to the physician examiners using a scale of 0 (low) to 10 (high). The physicians' mean ratings were: importance of the stations, 8.1 (SD, 1.8); success of the examination in testing core skills, 8.1 (SD, 1.6); and degree of challenge, 7.8 (SD, 2.1). The results indicate that a full-scale national administration of an OSCE for licensure is feasible using the model developed. Aspects of validity have been established and strategies to augment reliability have been developed.
To assess the effects of examinee gender, standardized-patient (SP) gender, and, in particular, their interaction on ratings made by SPs of examinees' interpersonal and communication skills in a performance-based examination of clinical competence.
The examination was administered to four classes of senior medical students (about 70 per class) at Southern Illinois University School of Medicine, 1988-1991. The skill dimensions tested were clarity of communication, thoroughness of explanation, professional manner, personal manner, and overall patient satisfaction. Split-plot analyses of variance were used.
There was no interaction of examinee gender and SP gender for any of the five rating scales. There was no main effect of examinee gender for four of the five scales; however, for personal manner, women students were rated slightly higher than men students. There was a main effect of SP gender, but the effect was not consistent from rating scale to rating scale or from class to class. Nevertheless, differences in ratings given by men and women SPs should not be of psychometric concern, since the ratings of men and women examinees are necessarily affected alike.
Except for the women examinee's higher performance in personal manner, the men and women examinees generally performed equally well with respect to interpersonal and communication skills, and they performed equally well regardless of the gender of the SP.
In previous studies assessing the effects of student gender, standardized-patient (SP) gender, and their interaction on multiple-station examinations of clinical competence, SP gender was confounded with cases, that is, male SPs were used for some cases and female SPs for others. The authors conducted two studies to investigate the effects of gender on a single case by using a male SP and a female SP for the same case.
Both studies involved one SP case in a comprehensive clinical evaluation used to assess fourth-year students at the end of a required ambulatory care clerkship at Saint Louis University School of Medicine in 1990-91 and 1991-92. In Study 1, 45 students in four rotations saw the same male SP; 42 students in three other rotations saw the same female SP. In Study 2, 69 students in six rotations were randomly assigned either the male SP (28) or the female SP (41) within the same rotation. For each study, to determine the interaction of student gender and SP gender as well as their main effects, analyses of variance were performed on the students' history-taking and physical examination scores and communication skills ratings.
Neither study showed a significant interaction of student gender and SP gender on history taking, physical examinations, or communication skills.
The findings of both studies suggest that the interaction between student gender and SP gender, unconfounded by case content, had no effect on the students' scores and ratings.
To determine who is the better rater of history taking in an objective structured clinical examination (OSCE): a physician or a standardized patient (SP).
During the 1991 pilot administration of an OSCE for the Medical Council of Canada's qualifying examination, five history-taking stations were videotaped. Candidates at these stations were scored by three raters: a physician (MD), an SP observer (SPO), and an SP rating from recall (SPR). To determine the validity of each rater's scores, these scores were compared with a "gold standard", which was the average of videotape ratings by three physicians, each scoring independently. Analysis included both correlations with the standard and a repeated-measures analysis of variance (ANOVA) comparing raters' mean scores on each station with mean scores of the gold standard.
Ninety-one videotapes were scored by the "gold-standard" physicians. Correlations with the standard showed no clear preference for MD, SPO, or SPR raters. ANOVAs revealed significant differences from the standard on three stations for the SPR, two stations for the SPO, and one stations for the MD.
An MD rater is less likely to differ from a standard established by a consensus of MD ratings than are SP raters rating from recall. If an MD cannot be used, an SP observer is preferable to an SP rating from recall.
We present herein data on US medical education programs and describe how medical schools are adapting to a changing health care environment. The data mainly derive from the 1995-1996 Liaison Committee on Medical Education Medical School Questionnaire, which had a 100% response rate. The data indicate that in the 1995-1996 academic year there were 91 451 full-time faculty members in basic science and clinical departments, a 1.6% increase from 1994-1995. In clinical departments, major increases occurred in emergency medicine (a 10.6% increase in full-time faculty) and family medicine (a 13.5% increase). Applicants for the class entering in 1995 numbered 46 591, an increase of 2.7% from 1994; however, the number of first-time applicants decreased slightly (0.6%). Of the 17 357 applicants accepted, 2179 (12.6%) were members of underrepresented minority groups. Health system changes are affecting medical school clinical affiliations. During the past 2 years, 42 schools saw a merger, acquisition, or closure involving medical school-owned or medical school-affiliated hospitals used for core clinical clerkships. At 15 sites, this change affected the distribution of students across clinical sites. In 1995-1996, 40 medical schools or their universities owned a health maintenance organization or other managed care organization, 93 schools contracted with a managed care organization to provide primary care services, and 96 schools contracted with managed care to provide specialty services. During the past year, 57 schools acquired primary care physician practices, and 70 started primary care clinics in the community.
Clinical skills assessment using standardized patients (SPs) is an accepted method of professional testing, but some factors can threaten the validity of the scores obtained.
The main objective of this work was to test whether the gender of the patients, the sequence of cases and the day of the examination have any impact on the scores of the examinees.
Data from examinations conducted at three Medical Schools are used. Each student encountered 10 different SPs (5 men and 5 women).
Analysis of variance was used to test the existence of association between the variables.
The results showed no significant differences or association between the scores and the three variables analysed.
The results are coherent with the studies that show that performance based examination using SPs can be used without introducing biases into students score.
This research was performed to study the attitudes that medical students hold concerning their relationships with patients, and whether such attitudes are gender-related, affect career plans, and influence their evaluation of psycho-social and biomedical issues.
One hundred fifty-three first year students at the Boston University School of Medicine completed the Patient-Practitioner Orientation Scale (PPOS), a scale that differentiates between a patient-centered vs. doctor-centered orientation toward medical practice, indicated their interest in community and primary care practice, and rank ordered psycho-social and biomedical clinical issues in terms of their perceived relative importance.
The data revealed that female medical students were more patient-centered, and that (across sexes) patient centeredness was positively associated with an interest in community and primary care practice and the ranking of psycho-social issues.
These findings indicate that differences in the practice attitudes of males and females exist very early on in medical training, and that these differences are associated with anticipated career choices. They also suggest that the PPOS may prove useful in measuring the attitudes of practicing physicians toward their clinical roles and might predict physicians' behavioral strategies and patient medical outcomes.
The Dundee Medical School has bean running OSCEs since 1977. In 1995, an integrated systems-based spiral curriculum on the core and options model was introduced. In 1997, outcome-based education was introduced as the basis for instruction, with a task-based educational strategy employed for students in years 4 and 5. This blend of educational strategies was considered in the design of the student assessment process. Assessment instruments, appropriate for use at each of the four levels of Miller's pyramid, were identified and included in the assessment process. The OSCE was used for summative assessment of students at the level of 'shows how' or simulation in years 2, 3 and 4. A year 2 OSCE is described here. Features of the Dundee OSCE are identified, relating to number and length of individual stations, practicalities or assessing a year group of students without student contamination with examination information and the blueprints used to design the examinations. Suggestions made for future development of the OSCE include the OSSE, the Objective Structured Selection Examination, and an exploration of the potential of the OSCE to assess attitudes, personal attributes and professionalism. The need is identified for a platform to debate issues such as should individual medical schools attempt to achieve national test centre standards with their examinations.
To investigate possible bias due to gender combination of students, role players and examiners in a high-stakes assessment.
Valid oral interactive contextualized examinations (VOICEs) is a long-station OSCE-style exam in general practice (GP). At the time of writing it consisted of 65% of the student's final GP mark.
In the VOICE, students undertake six tasks--four vivas and two role-plays. "Patient" roles are taken by professional role players who work regularly on the undergraduate curriculum. During the role-play, the student's clinical competence is assessed by an observing GP examiner. The communication skills marks are awarded by the role player and the examiner together, by negotiation. Data have been recorded detailing the role player's initial marks, the examiner's initial marks and their final (awarded) agreed marks for 1024 consultations.
512 final year medical students, 28 role players and 48 examiners. There were no inclusion or exclusion criteria. All those present on exam day became part of the data.
There was a significant relationship between gender and performance for some, but not all, stations. Correlations for multiple comparisons removed the significance. Female students perform better across the board than male students. While not always significant, this did affect grading. There was no significant association between the genders of role players and examiners with the question choices. There has been a significant worsening of male results since 1999. Differences exist in the way that pairs of mixed or single genders score students.
Objective structured clinical examination (OSCE) has been used to assess medical students since the mid 1970s, and in more recent years has been increasingly utilised to assess students from nursing and the allied health professions. This growing utilisation has led to considerable debate within the literature pertaining to the optimal use of OSCE as an assessment process. The purpose of this paper is to present a narrative review regarding some of the key issues affecting the utilisation of OSCE within the assessment of nursing students. The paper briefly reviews the historical development of OSCE within health professional assessment, and summarises some of its key strengths and limitations. It then offers a more 'in depth' consideration of the research literature pertaining to the reliability and validity of the OSCE process, which is then used as a basis for exploring some of the particular issues that need to be considered when OSCE is used to assess nursing students. Key issues identified include the need to carefully prepare and pilot new OSCE examinations and marking tools in order to ensure reliability and validity is optimised, and also the need to carefully consider the length, number and interdependence of OSCE stations to ensure that the potentially competing requirements of validity and reliability are balanced. The paper also recognises that whilst the evidence base regarding OSCE is extensive, the evidence base specific to nursing is more limited. There is therefore scope for further research in this area, as well as the need for careful debate regarding how national guidance may be a way of enhancing and standardising future OSCE examinations. The paper concludes that whilst caution must be applied in relying on OSCE as a sole means of practitioner assessment, used carefully it can make a helpful and meaningful contribution to health professional education.
- Rutala
- Colliver