Producing competent health workforce requires a robust assessment system to
validly measure graduates’ clinical competence. This study aimed to develop a valid
assessment tool to measure clinical competence at workplace and evaluate its effects on
clinical competence behaviors. A three-phase study was carried out using qualitative and
quantitative approaches. Phase 1 primarily focused on development of an assessment tool through narrative review, systematic review, Delphi technique and focus group discussion (FGD) with students. Phase 2, focused on validation of the assessment tool by examining response process validity from the users’ perspectives- students and clinical instructors (CIs). A 12-item questionnaire survey and FGD sessions was utilized to achieve this. Phase 3, implemented the assessment tool during a 4-week clinical placement, which produced data supporting internal structure, relation to other variables and consequences of the assessment tool. The qualitative data was thematically analyzed by coding protocol (open, axial and selective coding) while quantitative data by SPSS for descriptive statistics and reliability analysis. Phase 1, generated an assessment model guideline called CULTVATE and was used in designing a programmatic clinical portfolio (Pro-CP) as an assessment tool. Pro-CP consisted of Mini-CEX, DOPS and CREATE. The modifications made in phase 2 were 1) reduction of weekly caseloads to fortnightly due time factor, 2) modifications to the weekly and global rating scales, 3) increase training time, 4) provision of guideline for users 5) reduction of the assessment tool to pocket size. The modifications resulted in a revised assessment tool for phase 3, implementation. Feasibility study reported an acceptable completion time with ease of its use by CIs and students. High internal consistency of Pro-CP reported based on two clinical cases with Cronbach’s alpha value more than 0.8. Pearson correlation coefficient between Pro-CP and OSCE score was more
than 0.5, indicating moderate level predictive validity. Thematic analysis of FGD and in
depth interview data, generated 5 major themes related to perceived learning experience,
attitudinal change and behavioral change of the students. Theme related to perceived
learning experience was increased attitudinal behavior towards competence, with
categories relating to perceived assessment usefulness for learning and perceived ease of assessment use for learning. Theme related to attitudinal change was quality feedback
promotes clinical performance, promote student-centered learning and increased
perception of subjective norms towards competence development. Relevant categories
were task-level feedback improves clinical performance, application of constructive
feedback and normative believes towards Pro-CP. Theme related to behavioral change
were increased self-efficacy towards competence behavior, increased learning autonomy
towards competence behavior and need for institutional support. Categories were increased control belief towards behavior in practice, promote accountability towards learning autonomy, training and manpower resources, informing Pro-CP in curriculum and learning support. These qualitative results are evidence supporting Pro-CP’s consequential validity on learning experience, attitudinal and behavioral change. This study provided evidence to support validity of Pro-CP to content, response process, internal structure, relation to other variables and consequences. Hence, Pro-CP is a promising instrument that can be utilized by health professional educators to measure clinical competence of graduates