Figure 4 - uploaded by Andrew Cain
Content may be subject to copyright.
Source publication
Doubtfire is a web application that brings together an educational model based on constructive alignment and self-regulated learning to support students and educators. This model uses a task-oriented portfolio-based approach to assessment to scaffold and evidence different standards of achievement; backed by frequent formative feedback to support l...
Context in source publication
Context 1
... helps students further develop these skills by tracking progress through the unit tasks Woodward, Cain, Pace, & Jones A., 2013) and enabling students to visualise their progress toward achieving the unit learning outcomes (Law, Grundy, Vasa, & Cain, 2016). Doubtfire's student dashboard can be seen in Figure 4, this is the main view through which students interact with it. Student attention is focused on the task list, on the left of the screen, which provides an ordered list of tasks for the student to work through to achieve their target grade. ...
Citations
... and assessment in SIT723 was required. One proposition for addressing this issue was to use Task-Oriented Portfolio Assessment (TOPA) [16]- [18]. TOPA is a process-oriented assessment model. ...
... In Phase 1, students had workshops to learn more about Research Processes, Literature Review and Project Management, and the workshops further offered them opportunities to discuss with their peers and the unit team, specifics of their own project. The students were asked to submit following tasks on OnTrack -our web-based outcome assessment system [18], [32]. All the tasks on OnTrack were reviewed by the students' supervisors. ...
... and assessment in SIT723 was required. One proposition for addressing this issue was to use Task-Oriented Portfolio Assessment (TOPA) [16]- [18]. TOPA is a process-oriented assessment model. ...
... In Phase 1, students had workshops to learn more about Research Processes, Literature Review and Project Management, and the workshops further offered them opportunities to discuss with their peers and the unit team, specifics of their own project. The students were asked to submit following tasks on OnTrack -our web-based outcome assessment system [18], [32]. All the tasks on OnTrack were reviewed by the students' supervisors. ...
Students enrolled in software engineering degrees are generally required to undertake a research project in their final year through which they demonstrate the ability to conduct research, communicate outcomes, and build in-depth expertise in an area. Assessment in these projects typically involves evaluating the product of their research via a thesis or a similar artifact. However, this misses a range of other factors that go into producing successful software engineers and researchers. Incorporating aspects such as process, attitudes, project complexity, and supervision support into the assessment can provide a more holistic evaluation of the performance likely to better align with the intended learning outcomes. In this paper, we present on our experience of adopting an innovative assessment approach to enhance learning outcomes and research performance in our software engineering research projects. Our approach adopted a task-oriented approach to portfolio assessment that incorporates student personas, frequent formative feedback, delayed summative grading, and standards-aligned outcomes-based assessment. We report upon our continuous improvement journey in adapting tasks and criteria to address the challenges of assessing student research projects. Our lessons learnt demonstrate the value of personas to guide the development of holistic rubrics, giving meaning to grades and focusing staff and student attention on attitudes and skills rather than a product only.
... • Both assessment tasks had a hurdle requirement whereby students needed to achieve at least 50% in each of the assessments to pass the unit. This design, also referred to as 'outcome based assessment', is based on constructive alignment principles, that the final grade will reflect the achievement of all learning outcomes, as opposed to the 'measurement model' in which the final grade reflects an average or overall achievement (Cain et al. 2020). • The questions in both assessments were similar to those presented in previous years' exams and to problems presented in the weekly unit quizzes. ...
... Overall, students did better on the quiz assessment in comparison to the written assessment. An advantage of two shorter assessments with separate hurdles compared to a single exam with a single hurdle was that students were required to pass both parts of the unit material (Cain et al., 2020). This prevented a student passing who had obtained little learning in calculus, a topic that is fundamental to many future units. ...
... It was difficult to determine if a student had genuine uploading issues or were simply trying to get additional time to complete the task. In the next iteration of this unit, the assessment will be a range of smaller assessment tasks with students required to complete, to a satisfactory standard, all "pass" standard tasks in all topics to successfully pass the unit (Cain et al., 2020). ...
... TOPAM is a process oriented model, underlined by constructive alignment principles and outcome-based grading [18]. It consists of self-assessment against each learning outcome, frequent formative feedback, and delayed summative assessment [17]. ...
... Grading is delayed, until the end of the teaching period, so it can clearly reflect the level achieved by the student. Grading is also outcome based, reflecting the level of achievement at all dimensions rather than an average thereof [18]. ...
Promoting evidence-based assessment practices at scale across different disciplines is a challenge. Feedback, although powerful for affecting change in students, is not always adequately implemented. An assessment framework based on strong formative feedback and other evidence-based educational practices was designed to give capstone project students an
authentic learning experience, and was deployed across all our Software Engineering capstone project units involving more than 700 students and app. 30 academic staff members per teaching period. The scale of our setting raised the question about consistent and faithful implementation of the framework and its guiding principles, essential for achieving the assessment framework outcomes and equitability for students. In this study, we explore the challenges of implementing assessment practice at scale and report on our findings and support strategies in order to facilitate consistent engagement of teaching staff. The findings of our study form the building blocks of a decision-making framework to guide the implementation of ongoing pedagogical innovations at scale.
