Content uploaded by Ari Happonen
Author content
All content in this area was uploaded by Ari Happonen on Dec 23, 2021
Content may be subject to copyright.
Jari Porras
LUT School of Engineering Science
LUT University
Lappeenranta, Finland
jari.porras@lut.fi
Ari Happonen
LUT School of Engineering Science
LUT University
Lappeenranta, Finland
ari.happonen@lut.fi
Jayden Khakurel
LUT School of Engineering Science
LUT University
Lappeenranta, Finland
0000-0002-1397-5478
Abstract—Education is changing rapidly towards
digital and online forms in universities. E.g.
digitalization and Massively Open Online Course
(MOOC)s evolve remote education tools, which
changes the teaching approaches. Covid-19 accelerated
digital transition from classroom-based education
towards hybrid models, then online classes and finally
to fully online education. While the education has moved
from classrooms to Internet environment, the suitability
of this transition has not really been evaluated. This is
especially important to teamwork based courses in
which the collaboration within teams and between
teams is considered a major part of the learning
outcomes of the course. Our study presents experiences
on transforming a set of software engineering courses
from a fully onsite course to a fully online course and
lessons we have learned on that transition.
We analyze hackathon and capstone style courses
for traditional and remote execution modes. We have
used hackathons for decades and have experience of
different types of implementations. For capstones we
have two different courses that we have experience with.
Analyzing the team-based courses shows focal points
which area hard to solve with online approaches.
Online environment does not only make students
faceless to the teacher but also to other teammates. This
is especially visible in the long-lasting capstone
courses.
Keywords—online education, hackathon, capstone,
software engineering, teamwork, collaboration, digitalization
1. Introduction
Education is changing rapidly, and new teaching
methods emerge as the competencies of both teachers
and students evolve [1]. For some years, universities
have been emphasizing online education and/or online
tools supported courses as an integral part of their
teaching methods. For example, in our university the
two distinct campuses (some 200km apart) with shared
degree programmes require the use of online education
tools. In history, our university has for years promoted
the possibilities for time and location independent
learning due to a number of adult (master’s) education
programmes and high proportion of master’s students
working on their last years of studies (especially in
software engineering field). At the same time various
MOOC platforms set the pressure for quality
implementations of online courses and evolved tools
provide new possibilities. Online education was already
recommended and applied to courses that seemed to fit
best for online methodologies. Suddenly Covid-19
changed everything [2-3, 29] and all courses, regardless
of the applied teaching methods and base ideologies,
had to be considered to online teaching adaption.
This study reports our experiences of transforming
two different types of teamwork based courses on
software engineering education into online education
format and compare these online implementations to the
traditional implementations of the same courses. We
seek for answers to our research questions “How are
team-based courses affected by the fully online teaching
approach? and What should be taken into account to
provide similar learning experience for the team
members?”. Thus, we are particularly interested in the
challenges in teamwork aspects of these courses as these
courses emphasize the teamwork and other soft-skills
related to that as the main learning outcomes. Our
hypothesis is that the remote teamwork will make the
collaboration harder. We approach our research
questions by thematically analyzing the reports from
each course (qualitative data rather than quantitative due
to small number of courses in our study).
2. Related work
The impacts of online education have been studied
in the literature from different perspectives. Bernard et
al. [4] did meta-analysis of the comparative distance
education (DE) for 1985-2020 literature to analyze
independent achievement, attitude, and retention
outcomes. The findings strongly suggested that many
remote education applications outperform their
classroom counterparts and that many also perform
poorly too. Dividing achievement outcomes into
synchronous and asynchronous forms of DE produced a
somewhat different impression. In general, mean
achievement effect sizes for synchronous applications
favoured classroom instruction, while effect sizes for
asynchronous applications favoured DE.
In 2016, Chen et al. [5] compared the achievements
and learning experiences of 192 onsite and online 311
Experiences and Lessons Learned from Onsite
and Remote Teamwork Based Courses in
Software Engineering
Chinese students participating in a Massively Open
Online Course (MOOC). The study found, i) onsite
learners had a lower attrition rate with learning
performance than the online students, ii) learners who
had completed all their learning assignments, no
significant difference was detected between onsite and
online participants’ average assignment scores and they
had equally likelihood to win learning awards, iii) based
on their learning experiences, no significant difference
was detected between the online and onsite students’
ratings of technology quality and usability instructional
content, and the design of learning assessment.
Ali [6] adopted a meta-analysis methodology, and
pertinent literature review captured the essence of
continued learning during these unprecedented times.
The study illustrated that, i) universities worldwide
move towards online learning or E-Learning, ii) apart
from resources, staff readiness, confidence, student
accessibility and motivation play an essential function
in ICT integrated learning. Based on the findings,
authors proposed use of technology and technological
gadgets by the staff members to enhance learning,
especially in exceptional times. Moreover, they pointed
online and remote learning as an expected necessity in
covid-19 like pandemics, which require lockdowns and
social distancing. Also, Olasile and Soykan [7]
summarized the challenges and opportunities raised by
the Covid-19 pandemic towards online learning serving
as the educational platform. The authors concluded that
previously adopted online learning is different from
emergency remote teaching. At the same time, the
authors pointed online learning to be more sustainable
and instructional activities will become more hybrid
based on the experiences gained during the pandemic.
3. Teamwork based courses
Teamwork is well studied in academic literature,
but still employers require universities to do more to
improve graduate’s readiness to work in team-based
environments [8]. This should be easy as almost any
course may apply teamwork [9] or team assignment [10]
as a part of the course completion. Sometimes, adding
teamwork for a course is a tool for teachers to ease up
their workload. For example, on a large course (e.g.
500+ students on some Fundamentals of Software
Development course) grading only 80 assignments of 5
students compared to 400 individual assignments is
definitely a much less demanding task. In this approach
teamwork is typically not the core learning outcome that
is evaluated. There are, however, courses that
emphasize teamwork and other related soft-skills as the
main learning outcomes [11]. In software engineering
capstone courses [12] and hackathons [13] typically rely
on teamwork skills and other related soft-skills [14, 40
].
Capstones are heavily used as the “final project” of
the graduates in software engineering (Merriam-
Webster defines capstones as “the high point: crowning
achievement”). Capstones have been analyzed from
various perspectives. Palacin et. al studied the
implementation of the capstones from the design
thinking perspectives in [17] and the extension of the
capstone by sustainability content in [18]. Khakurel and
Porras [19] analysed the soft skills learned in a
developing capstone course with a heavy emphasis on
real world projects and with actual industrial customers.
Hackathons [20] is second teamwork based course
emphasized in this paper. The implementation and
definition of hackathons were closely studied in a recent
literature review [21]. The outcome of that literature
review was that the hackathon represents a rather short-
time event dedicated to solving some challenge, usually
with some industry connection [22, 38-39] or in specific
university - industry collaborative setting [31], for
example to use hackathons to boost innovations as
university - industry collaboration activity [23]. The
three most emphasized learning outcomes of hackathons
presented in [21] were a) creativity or innovation skills,
b) teamwork or collaboration skills and c) some context
specific skill (e.g. Java programming skills). Based on
the previous studies, hackathons are not so widely used
as a part of education although they could provide
similar types of learning outcomes capstones are
intended to have [23]. As such hackathons are seen more
as an informal learning platform [25]. Hackathons have
many synonyms like code jams or code fest [26] but the
term, code camp, has earlier been used for hackathon
type of events used for educational purposes [27][28].
Reports from hackathons often document the challenges
tackled, formats of the events and not so much the
learning outcomes. Overall, both hackathons as well as
capstones rely heavily on teamwork effort. While in
hackathons, time limitation requires efficient teamwork,
in capstones the project size is the main limiting factor.
In both cases efficient communication among the team
and with the external stakeholders (other hackathon
teams and customer in capstones) is crucial for the work.
4. Courses in our study
In this section we describe shortly the courses under
study. Both types of the courses have strong emphasis
on teamwork and as such the transition from onsite to
online courses has had some major consequences.
4.1. Capstone type of courses
We have selected two different capstone type of
courses with a bit different learning objectives.
4.1.1. Software and application innovation (Course
1). Software and application innovation course is a
semester-long (14 weeks) project type master’s level
multidisciplinary course with both Finnish as well as
international students from software engineering,
industrial engineering, and business administration
programmes. This course is built around teamwork and
related soft skills and the course has been run for more
than 10 years. First implementations were fully onsite,
classroom teaching based, with a set common lectures,
active weekly team-wise tutoring and public
presentations. One of the key activities in the beginning
of this course is the theme based collaborative analysis
and innovation session that provides teams the ideas to
start with. For example, last few years the theme of the
course has been linked with the UN sustainable
development goals and how digitalization can be used
to tackle those. Students with different backgrounds
(discipline, culture) will have a good chance of having a
broad view to the challenges before starting the actual
innovation processes. Teaming up process is also
important so that the students find team members that
are interested in the similar topic. One could argue that
in industry professionals don’t pick their teams at work
and because of that students shouldn’t pick their teams
at school, but to our experience many startups are
formed around people who want to work together and
develop something they share same interest. All in all,
both ways of forming teams have their pros and cons.
As the number of common classroom activities in
the first implementations of the course were rather
limited, the course added a set of online meetings some
5 years ago to enable better linking of different teams.
These online meetings were common to all teams and
required teams to prepare some presentations and be
active with the other teams. Thus the course transformed
into a hybrid onsite-online model for 2014-2019. Course
on fall 2020 was implemented fully online due to Covid-
19 situation. This course has strong cultural flavor due
to the different nationality and cultural backgrounds of
the students and use of the global challenges as topics.
4.1.2. Running a software project (Course 2).
Running a software project course is a semester-long
(14 week) master’s level capstone course for software
engineering students (equal amount of Finnish and
international students). The evolution of this course has
already been reported in [17],[18] and [19] from the
skills, content and teaching methods perspective, but
COVID-19 forced the implementation to change from
onsite to online model. The course was a traditional
project based course with weekly lectures and tasks that
help the teams to advance their projects with the real
customers [19]. In this course a set of customers and
their challenges are provided to teams and students form
the teams by selecting what they want to do. Typically,
this course has 40-60 students forming groups of 4-6
persons. The main point of our capstone is that the
students need to listen to the customer rather than
innovate their own solution [19]. Like with the software
and application innovation course the students have few
common steps in which the teams present their work to
other teams, e.g. internal testing of the prototypes or
public testing days. This course requires teams to use
some internal workspace that will have proper interfaces
to the customer and to the teachers (to follow the team
activities). The Covid-19 situation changed the spring
2020 course to be a hybrid model (teams managed to
have face to face meetings among the groups and with
the customers) and spring 2021 a fully online course (all
communication happened through online environment).
4.2. Hackathons
Our university has been one of the first universities
to use a hackathon type of event as an educational
approach and also as a tool for university - industry
collaboration [23]. In the educational context
we traditionally have been arranging these events onsite
during exam weeks or intensively during a
weekend. The currently dominant industry focus in
hackathons existed partly already in our original events
but the emphasis was on the learning of students [32],
coding with software language X and studying a new
technology Y, and not on the benefits of the company
[33]. We have selected a few implementations of our
events from the long history [24], to illustrate different
implementations of these events.
4.2.1. .NET Code Camp. .NET code camps were a
week long master’s level hackathons arranged on fully
onsite mode. The events were pretty typical hackathons
with a challenge setting, lectures, innovation, teaming,
development and presentation phases. Length of the
events were 5 to 7 days and a computer classroom was
dedicated for the course. The events were arranged
outside the normal lecturing periods (e.g. intensive
course weeks, exam periods, etc.) and students were
expected to participate actively in the course by staying
in the selected facility. Our slogan for the events “no-
one is left alone” was also highly visible in the
evaluation of the outcomes as the “hackathon spirit” was
worth 20% in the evaluation. All this with the goal of
learning some new technology (e.g. .NET platform with
a link to the Microsoft Imagine Cup) resulted in a very
active participation and collaboration within the
course. These events were highly education oriented and
as such students participated in the events like any
mandatory lectures. Students felt that these events
provided them new hard skills in addition to the soft
skills normally emphasized in hackathons. These events
were open for any software engineering student and they
attracted both Finnish as well as international students.
Team sizes in hackathons are limited to 3-4 as based on
our experience that small/large group enables efficient
peer support within and between teams (one member
can always discuss with other teams which is one of the
key elements for successful operation).
4.2.2. ICT4S Hackathon. ICT4S Hackathon was a
summer 2019 special university - company
sustainability and AI/ML focused 1-2 weeks long
hackathon event. The event was an international
master's level hackathon aimed for the last year's
students. Two implementations of the hackathon were
provided. Some students took a shorter one week
version and only participated in one challenge, while
some others took a full 2 weeks and contributed to two
separate challenges. The company providing the
challenges, offered real case data and area specific
knowledge for the students, remote startup lecture,
expert database lecture, Q&A session and questioning
and mentoring hours (all remote). With the
collaboration in the hackathon, the company was
seeking to develop new business opportunities [15],
build long lasting digital age collaboration models with
the university [37] and to start changing their
organization culture to be more open [35]. The event
had 2 different remote locations offering local onsite
sessions simultaneously (multiple facilitators were
used) as well as 100% remote participation. The final
results presentation was 3 hours session. Best
presentation was selected to show the results for ICT4S
conference audience as a hackathon result presentation.
In this hackathon some of the non-typical parts
were: 1) option of new students to join to teams in the
middle of the event (1 week participants to full length
participating teams), 2) typical session was organized in
evening hours thus providing possibility to students
with a summer job, 3) various mentoring session (expert
lecture, open Q&A session, mid hackathon
presentations for the teams to receive direction guidance
from the company) and 4) multiple injection of self
learning additional support materials from the lead
teacher. Students had the right to study 100% remotely,
but it was also possible to participate in the ICT4S
conference lectures and meet the conference experts.
4.2.3. DevOps Hackathon. DevOps development
hackathon was arranged in early autumn 2020 as a fully
online event with recorded expert lectures and online
participation. Well known remote conferencing tools
TEAMs and Zoom were used in the arrangements. This
particular hackathon was a longer event (3 weeks) than
traditional hackathons as activities were mostly in
evening and weekend times. This event could even be
called flipped-hackathon as the students were provided
the necessary knowledge of DevOps (recorded expert
lectures) and their task was to build the actual course.
As such, this was one of the first hackathons where
students themselves had the opportunity to develop the
course content for the software engineering master
programme and all this needed to happen online.
5. Discussion and lessons learned
The selected set of courses are compared (See Table
1 and Table 2), from a teamwork perspective, on
different modes of implementation (fully onsite, hybrid,
fully online). In addition to team reports (that also
consist of student reflection), each course collects
student feedback on the course implementation issues.
Both of these sources were used to collect the student
perceptions of the impacts of transition to online
education. The following tables compare the course
implementations based on a) lectures (or lessons), b)
innovation activities, c) team forming, d) internal team
activities (within teams), e) external team activities (e.g.
towards teachers and customers), f) presentations.
Table 1. Capstone course Implementation description and comparison
Course 1 Onsite Course 1 Hybrid Course 1 Online Course 2 Onsite Course 2 Online
Lectures Students had to be present in the classroom,
so it was hard to start the course, if a student
missed the lectures.
It was easy to activate students.
Students were active in class and with the
natural setting for working in small groups.
Easy to pinpoint questions and get all
engaged.
Possibility for flipped
classroom activities.
Recorded videos provided
more time for team
activities, not all watched
those beforehand.
Difficult to observe
students and activity of the
students was weak.
Same as in
Onsite/Hybrid course
1.
Same as in Online
course 1.
Improved use of
technologies in 2021
improved outcomes.
Innovation Classroom activity in small groups with the
use of whiteboards to share ideas.
Ideas spread across teams, easy to continue
another group’s idea
Hard to separate friends into different
groups.
Innovation action in small
randomly generated groups
in Zoom using shared Miro
board for innovation.
Easier to allocate strangers
together.
Teamwise innovation as the topics are provided
by customers. Teams needed to discuss with the
customers and find their requirements. Teacher
divided topics based on priorities. Complaints of
not allowed to have their own topic.
Teaming Teaming in the classroom with
requirements to have both technology and
business people. Easy to locate talents for
the team as everybody may discuss at the
same time
No clear process how to
team up. Students in the
random groups mostly
stayed together.
Teaming in the
classroom. Some
predefined groups.
Students formed
teams by themselves
and informed the
teacher.
Internal Teams should select a suitable workspace for themselves. Different cloud
services have been used.
Team needed to set internal workspace and
communication channels.
External Only final
presentation. No
knowledge of what
the others were
doing. Request for
common
workshops.
Possibility of onsite
tutoring for the teams
was semi-actively
used. Online
presentations and
discussions with the
other teams.
Complaints that
these should have
been onsite rather
than online.
Possibility for online
tutoring was rather badly
used.
Online presentations and
discussions with the other
teams. No complaints of the
online activities as
mandatory and all others
were also online.
Teams managed and
documented their
customer meetings
(teachers wasn’t
always there). Weekly
onsite tutoring sessions
with teachers. Other
teams were activated
by having in-class
testing days and public
testing days.
Teachers were
invited to the online
customer meetings
which helped in
following the teams.
Tutoring meetings
were held online
Presentations Onsite presentation day extremely active in
discussion and feedback. Easy to follow the
student focus.
Online presentations do not
initiate questions. Hard to
see student’s activity level.
Onsite presentations,
teams actively
commenting
Quality of the
presentations
decreased, hard to
get any comments
Capstones in an online environment shows both
positive as well as negative effects to the students.
Participation to lectures seems to be easier as lectures
are (pre)recorded and provided for the students. This
kind of flipped-classroom activity requires more control
as students do not necessarily watch the videos
beforehand. Some students think that they may start the
course whenever they wish and there might not be
suitable groups for them (they become aliens in a group
if they have not been there from the beginning). Whole
class activities (group to group communication) is easier
in an onsite event as current tools like Zoom and Teams
do not really support multiteam discussions (for
presentations they are fine). In general, the innovation
activities are easier in a shared space (miro provides one
tool but misses the discussion element). Internal team
discussions are well supported with different tools, e.g.
teams, but the proper use of them required some
experiences (i.e. the fully online 2021 capstone course
was much bigger success, than hybrid 2020 course due
to experiences of both teachers and students from the
first covid-19 season and thus better operation modes).
Table 2. Hackathon course comparison
Traditional Hack Hybrid Hack Online Hack
Lectures Classroom activity in the beginning
of the hackathon (and few lessons
every now and then). Common
lectures got the students motivated to
course work.
Presence (onsite, online) in opening and closing
sessions recommended, recording of all technical
sessions recommended, dealing with different
types of teams (onsite, online) challenging, lengths
of sessions should be carefully considered
(especially online)
In online streamed form, pre-
recorded and recorded online to
give students multiple review
times options
Innovation Typically the end of day one was
used for common innovation with
team results focus. Teams did
discuss with each other and helped to
fine tune the idea so that the
implementation could be started with
clear plans.
Provides a wide variety of possibilities but if real-
time multi location event that has its own
challenges, presentations can be controlled in
online environment more efficiently
Invisibility of student activities
to the teacher, proper tools need
to be provided, difficulty to
students to get the hackathon
feeling
Teaming In the first traditional hackathons the
teams were mostly pre-formed or
were formed easily in the class as
most of the students knew each
others (as they all represented
software engineering).
Teaming like in traditional hacks in local physical
event, opening day needs to create the hackathon
and team spirit locally and across network, use of
multiple sites provides possibilities but local event
better for socializing
Pre-teaming and “on-site”
teaming possible, getting to
know others and team members
challenging, pre-made profiles
for each participant
Internal Open and commonly used Wiki site
has been used for both internal as
well as external communication. The
original idea “no-one is left alone”
required open communication and
sharing of knowledge. Students
appreciated the doing and learning
together approach or so called “code
camp spirit”.
Teams were formed with the
idea that 2 team members can work
together and the third or fourth can
discuss with other teams. Lots of
discussions happened face to face
though some persons were much
more active than others [37].
Discussions happened in the
classroom but some common issues
were collaboratively documented to
the Wiki.
The team communication is tried to be supported
in a hybrid model by arranging physical spaces
where the teams can meet, if they do choose to do
so. It has been noted previously, that e.g. if the
University premises have extras like free
breakfast, air conditioner, good and affordable
lunch easily available and so on and the students
feel they do not have all these options available
they do prefer to use the premises and steer
towards physical meetings. On the other hand,
many students did comment on their feedback, that
it is really easy just to sit in home and use remote
tools, if the meeting does not offer any “perks”. So
it seems students do not seem to think that working
F2F itself is not good enough, they need
something extra to justify the time and effort
needed to e.g. move from home to University.
Technologies for internal team
communication available,
online meetings easier to
“omit”, huge variations in the
skills, teamwork spirit,
readiness to flex on own time
tables and willingness to be
available “all the time” in online
hackathons, difficult to measure
“code camp spirit” or “no one is
left behind” attitude
External Students have best team level external
communication experiences when these are base
organized by the mentoring teacher, Challenge of
communication between remote and local teams,
recording of shared sessions valuable,
Availability of external resources important
Facilitation and mentoring
support for external
communication is quite crucial
for team success, Frequent Q&A
and technical sessions needed.
Presentations All presentations and demonstrations
happened in the classroom and was
the highlight of the hackathon.
Students wanted to show their best in
this event.
Challenge of presentation in multi-location events,
engaging all students important, local activities,
connections should work in multi-location events,
lack of common spirit, considering roles of team
members
Pre-recorded presentations
suggested, Multiple
presentations for feedback and
dissemination. Students value in
situ feedback more than
recording / delayed feedback.
The transition from fully onsite course / events to
fully online events has taught as few valuable lessons.
In these lessons we aim to emphasize both teacher and
students perspectives. These lessons are based on
thematic analysis of the student reports and teacher
experiences from these courses. The analysis is of
qualitative nature due to small sample size.
Online environment hides feelings and make
people look like “neutral”
This lesson can be seen in all of the activity
categories. In lectures and presentations the online
environment makes it impossible for the teacher to read
the listening students’ faces or impressions. Highly
talented teachers can get some specific reactions out of
the student group but getting into the individual level is
extremely challenging. In team building activities
students have difficulties in getting familiar with the
other students and as such finding proper teams
becomes harder. This also may result in teams in which
everybody is not equally committed or people do not
continue up until the end of the course. Teachers can
have per team (remote) discussion sessions to alleviate
these challenges. Our guidance to this challenge is to ask
the students keep their cameras on and pinpoint
questions to specific students. For example, in our
capstone courses the students have roles in their teams
and it is possible to address “customer interface
responsible” to answer the question. Thus, students can
answer questions that are related to their responsibilities
and other teams get to know the other persons in the
similar role. As such this can be tackled partly within
online environment.
Online environment decreases visible activity
In an online environment the visible student activity
decreases considerably. This is partly due to the
“faceless” nature of the online environment (lesson
one). It seems to be easier to be quiet than ask questions.
It may also be possible to focus on other things than the
required activity (this is also case in online meetings
etc.). This would be harder in onsite events. In
teamwork activities the inability (either teacher or
fellow students) to follow contributions may lead to a
badly working team. In the latest online implementation,
the students were asked to record their customer
meetings. As meetings covered different aspects it was
easier to see if the responsible persons were doing their
share of the work. Our guidance to this is that each team
member should have clear role and responsibilities that
can then be followed (lesson one). Most of the activity
is not necessarily visible in real time communication but
the teams may use their internal working areas (like
Teams, GitHub etc.) that enable visibility of different
actions.
Online environment may shift responsibility
While an online environment provides a set of
possibilities to teachers and students, e.g. pre-recording
lectures or recording, these new possibilities shift some
responsibility to the student. Like seen in our courses the
existence of a lecture recording does not mean that the
students watch that. Teachers may require a set of in
progress content deliverables to keep the work in flow
and to give the teacher the option to use continuous
evaluation means of giving final grade. In courses that
are practically oriented, e.g., capstones and hackathons,
the work should be divided in phases and each phase
needs to have a deadline. This enables students to
schedule their work and the teachers see that the work
progresses. The phases and communication of schedules
are thus extremely important.
Online environment decreases team to team
communication
Our experiences show that the online environment
makes the teamwork more as a separate team rather than
a peer to the other teams. Team to team communication
has been an important element of our team based
courses and the online environment seems to miss
elements that would promote this activity. The students
of the software and application innovation course
requested especially this element and it does not seem
to fully work online. On the other hand, teams may
communicate but that part is not visible to the
teacher. We see this as the major challenge that is not
fully supported by any tool. Tools like Zoom and Teams
provide an interface for mostly one-sided
communication, for example presentation, while
interactive communication is not possible. Use of
breakout rooms has been used in our testing events,
when participant amount was around 20 or more. Miro
board and wikis provide ways on interactive planning
and writing but the missing communication part leaves
this as a major drawback in online implementations.
Challenging hybrid approach
The hybrid approach can be categorized in two
types; courses that have both onsite and online
education (not at the same time) and courses that have
both local and remote education at the same time. The
software and application innovation course is an
example of the previous. The challenge in this course is
that students would like to have only onsite sessions as
they feel those are more natural and communication
between teams is easier. On the other hand, then
students would have to be at the university all the time,
which would make remote teams impossible, but
students can be on-campus but dispersed.
The ICT4S hackathon is an example of the second
type of hybrid courses. The main challenge in this
hybrid approach is that it needs to be “script” for both
the traditional and for the online way, this is especially
demanding if some students are 100% remote
participating in a hybrid hackathon, which demands the
teacher to organize almost everything in both ways.
Somewhat easier case is the model where the teacher has
traditional modules and remote modules inside one
hackathon event. To our experience, when a hybrid
model is successful, it is one of the most involving
models which give e.g. companies most minutes to be
part of action, is least restricting in a sense how students
can participate and is most flexible on getting differently
minded people to enlist to a hackathon event. The
flipside is that the hybrid format can be the most
demanding to master and organize elegantly. For teams
the hybrid multi location model is also challenging as
one easily communicates locally and forgets the online
participants. Our advice is to have separate organizing
teams in both ends so that there are both local parts as
well as common multi-premise parts.
Suitable tools exists but operation models are
still evolving
Digitalization has given us the needed tools (e.g.
Zoom, Teams, Miro, wiki, Discord, slack, Teams, etc.)
to move towards hybrid models and fully onsite models
but we lack the proper processes to utilize them in all
activities. We have observed the instructors to be in
need to have more data on intra- and inter-team
collaborations, especially in agile workflows. The same
is true for both the teachers as well to the students. The
experiences of several implementations of online
courses (both capstones as well as hackathons in 2020
and 2021) have shown us that both teachers as well as
students have slowly adjusted to the “new” operation
mode. Teachers have learned not to copy their
traditional lectures to online environment but to use
flipped classroom approaches, mixed working methods
and shorter sessions (90-minute lecture over zoom
would be suboptimal learning method for most). On the
other hand, the students have learned to take some
responsibility of their learning. More of the students do
watch the videos beforehand or do the pre-tasks.
Students have also found the best ways for them to stay
connected. Thus, the success of this transition towards
online education even in teamwork-based courses is
highly dependent on the evolution of working habits of
both teachers as well as students. So far, we have
succeeded in engaging the students with the tools and
processes in most of the activities. The biggest challenge
is to enable larger scale discussion between all teams.
6. Conclusion
The study reflects our experiences on transition
from onsite teamwork-based courses to a fully online
version of the same courses. We have a long history of
team-based courses in our software engineering
education and we wanted to reflect how the change of
the course delivery affects teamwork and its evaluation.
Both the traditional capstone type of courses as well as
educational hackathons from master’s level were
evaluated, which revealed that both onsite as well as
online education has some clear benefits. In onsite
implementation the teaming and partly the innovation is
easier for the students and activation of the students
more visible to the teacher. Online implementation
provides better possibilities to the lectures as teachers
may use live online lectures or record the lectures
beforehand (for flipped classroom) but decrease the
visibility to the students and their activation. The
internal teamwise work might not suffer in online
education but the team to team collaboration decreases.
This is especially destructive in hackathon type of
courses that are based on wide collaboration.
These experiments are based on reflections on the
lessons learned so far. Collaboration environments and
tools are evolving and provide us tools to implement
online activities. Now we need proper ways to utilize
these tools, e.g. new ways to activate companies to
collaborate with universities [38] and students to
teamwork within their teams and between their teams.
We should also gather more knowledge what drives and
motivates [36] non university students to participate to
open hackathons organized as online courses and seeks
to understand deeper the success in digital innovation
efforts [30]. Additionally, we assume that in post covid-
19 new normal, many campuses will go back to face-to-
face teaching as primary teaching method, albeit with
additional online/hybrid contact hours.
References
[1] Garcia-Huidobro, J.C., Nannemann, A., Bacon, C.K.,
Thompson, K. (2017), “Evolution in educational change:
A literature review of the historical core of the Journal of
Educational Change”. J Educ Change 18, pp. 263-293
[2] Alterri, D., Hindi, M., AlMarar, R., Shubair, R.M.
(2020), “Transition to distance learning during the
COVID-19 pandemic: Efforts within the Higher
Education sector in the United Arab Emirates”, Journal
of Applied Learning and Teaching, Vol 3, No 2, pp. 1-9,
doi: 10.37074/jalt.2020.3.2.17
[3] Basilaia G, Kvavadze D. Transition to Online Education
in Schools during a SARS-CoV-2 Coronavirus (COVID-
19) Pandemic in Georgia. PEDAGOGICAL RES. 2020;
5(4):em0060
[4] Bernard RM, Abrami PC, Lou Y, et al., (2004), “How
Does Distance Education Compare With Classroom
Instruction? A Meta-Analysis of the Empirical
Literature.”, Review of Educational Research, 74(3), pp.
379-439, doi:10.3102/00346543074003379
[5] Chen, W., Jia, J. (2916), Comparison of Online and
Onsite Students’ Learning Outcomes and Experiences in
a Massively Open Online Course in China, JETDE, Vol.
9, Iss. 1, Article: 5, doi: 10.18785/jetde.0901.05
[6] Ali, W, “Online and Remote Learning in Higher
Education Institutes: A Necessity in light of COVID-19
Pandemic”, Canadian Center of Science and Education.
2020. doi: 10.5539/hes.v10n3p16
[7] Olasile, A., Soykan, E. “Covid-19 pandemic and online
learning: the challenges and opportunities”, Interactive
Learning Environments, (2020) doi:
10.1080/10494820.2020.1813180
[8] Riebe, L., Girardi, A., Whitsed, C. (2016), "A Systematic
Literature Review of Teamwork Pedagogy in Higher
Education", Small Group Research, Vol. 47, Iss. 6, pp.
619-664, doi: 10.1177/1046496416665221
[9] Hall V. (2002), "From Team Work to Teamwork in
Education", In: Leithwood K. et al. (eds) Second
International Handbook of Educational Leadership and
Administration. Springer International Handbooks of
Education, Springer, 2002, Vol 8, pp. 697-733, doi:
10.1007/978-94-010-0375-9_24
[10] Miles, J.A. (2020), "The Influence of Goal Commitment
and Partner Status on Team Assignment Outcomes",
Psychological Reports, Vol. 123, Iss. 3, pp. 844-871, doi:
10.1177/0033294118821299
[11] Raibulet, C., Arcelli Fontana, F. (2018), "Collaborative
and teamwork software development in an undergraduate
software engineering course", Journal of Systems and
Software, Vol. 144, pp. 409-422
[12] A.J. Dutson, R.H. Todd, S.P. Magleby, C.D. Sorensen
(1997), A Review of Literature on Teaching Engineering
Design Through Project‐Oriented Capstone Courses,
Journal of Engineering Education, Vol. 86, Iss. 1, pp. 17-
28. doi: 10.1002/j.2168-9830.1997.tb00260.x
[13] Komssi, M., Pichlis, D., Raatikainen, M., Kindström, K.,
Järvinen, J. (2015), What are Hackathons for?, IEEE
Software, Vol. 32, Iss. 5, pp. 60-67, doi:
10.1109/MS.2014.78
[14] Joint Task Force on Computing Curricula IEEE
Computer Society Association for Computing
Machinery, “Software Engineering 2014: Curriculum
Guidelines for Undergraduate Degree Programs in
Software Engineering,” Jt. Task Force Comput. Curricula
IEEE Comput. Soc. Assoc. Comput. Mach., no.
February, p. 134, 2014.
[15] Happonen, A., Nolte, A., Bystriakova, N., Santti, U.,
Kärhä, K. (2021), Hackathons for New Innovation Seed
and Business Model Development Needs in
Digitalization driven Sustainability, Circularity and
Environmentally Friendly Solutions Demanding
Digitalizing Societies, In Book: New Innovations in
Economics, Business and Management, p. 23
[16] J. Ikonen, A. Happonen, J. Porras (2007), “Game
programming for learning - Experiments in teaching
protocols with game programming”, In proceedings of
the Proceedings of the 18th EAEEIE Annual Conference,
Praha, Czech Republic, p. 9
[17] Palacin-Silva, M., Khakurel, J., Happonen, A.,
Hynninen, T., Porras, J. (2017), Infusing Design
Thinking Into a Software Engineering Capstone Course,
The 30th IEEE CSEE&T Conference, pp. 212-221, doi:
10.1109/CSEET.2017.41
[18] Palacin-Silva, M. V., Seffah, A., & Porras, J. (2018).
Infusing sustainability into software engineering
education: Lessons learned from capstone projects.
Journal of cleaner production, 172, pp. 4338-4347
[19] J. Khakurel and J. Porras, "The Effect of Real-World
Capstone Project in an Acquisition of Soft Skills among
Software Engineering Students," 2020 IEEE 32nd
Conference on Software Engineering Education and
Training (CSEE&T), Munich, Germany, 2020, pp. 1-9,
doi: 10.1109/CSEET49119.2020.9206201.
[20] Happonen, A., Minashkina, D. (2018), "Ideas and
experiences from university industry collaboration:
Hackathons, Code Camps and citizen participation",
LUT Reports series report 86, pp. 1-21, doi:
10.13140/rg.2.2.29690.44480
[21] Porras, J., Knutas, A., Ikonen, J., Happonen, A.,
Khakurel, J., Herala, A. (2019), “Code camps and
hackathons in education - literature review and lessons
learned”, HICSS 2019, Hawaii, USA, pp. 7750-7759,
doi: 10.24251/hicss.2019.933
[22] Valença G., Lacerda N., Rebelo M.E., Alves C., de Souza
C.R.B. (2019) On the Benefits of Corporate Hackathons
for Software Ecosystems – A Systematic Mapping Study,
PROFES 2019. Lecture Notes in Computer Science, Vol.
11915, pp 367-382, doi: 10.1007/978-3-030-35333-9_27
[23] A. Happonen, D. Minashkina, A. Nolte, M.A. Medina
Angarita (2020), Hackathons as a company – University
collaboration tool to boost circularity innovations and
digitalization enhanced sustainability, AIP Conference
Proceedings, Vol. 2233, Iss. 1, pp. 1-11, doi:
10.1063/5.0001883
[24] Porras, J., Khakurel, J., Ikonen, J., Happonen, A., Knutas,
A., Herala, A., Drögehorn, O. (2018), Hackathons in
software engineering education: lessons learned from a
decade of events, In Proceedings of the 2nd International
Workshop on Software Engineering Education for
Millennials, pp. 40-47, doi: 10.1145/3194779.3194783
[25] Nandi, A., Mandernach, M. (2016), Hackathons as an
Informal Learning Platform. In Proceedings of SIGCSE
'16, pp. 346-351, doi: 10.1145/2839509.2844590
[26] Briscoe G.: Digital innovation: The hackathon
phenomenon, (2014)
[27] K. Heikkinen, J. Porras, and J. Ikonen, “Introducing code
camp process,” in IMSCI 2008 Proceedings, 2008, Vol.
2, pp. 175-180
[28] J. Porras, K. Heikkinen, and J. Ikonen, “Teaching .NET
programming with code camp approach,” in IMSCI
2008 Proceedings, 2008, vol. 3.
[29] Happonen, A., Tikka, M., Usmani, U. (2021), A
systematic review for organizing hackathons and code
camps in Covid-19 like times: Literature in demand to
understand online hackathons and event result
continuation, In 2021 International Conference on Data
and Software Engineering (ICoDSE), p. 6
[30] Kitsios, F., Kamariotou, M. (2019). Beyond open data
hackathons: Exploring digital innovation success,
Information, Vol. 10, Iss. 7, Artcle: 235, pp. 1-10, doi:
10.3390/info10070235
[31] Steglich, C., Marczak, S., Guerra, L., Trindade, C., Dutra,
A., Bacelo, A. (2021), An Online Educational Hackathon
to Foster Professional Skills and Intense Collaboration on
Software Engineering Students, In SBES '21: Brazilian
Symposium on Software Engineering, pp. 388-397, doi:
10.1145/3474624.3476973
[32] Happonen, A., Minashkina, D. (2018), "The Code Camp
Story: Facilitators and Participants View into University
– Company Collaboration", TYYLI Blog, pp. 1-2, doi:
10.13140/RG.2.2.12329.95843.
[33] Happonen, A., Minashkina, D. (2018), “Ideas and
experiences from university industry collaboration:
Hackathons, Code Camps and citizen participation”,
LUT Scientific and Expertise Publications report 86, pp.
1-21, doi: 10.13140/rg.2.2.29690.44480
[34] Sadovykh, A., Naumcheva, M., Khazeev, M. (2020),
Hackathons as a Part of Software Engineering Education:
CASE in Tools Example, Lecture Notes in Computer
Science, Vol. 12271, pp. 232-245, doi: 10.1007/978-3-
030-57663-9_15
[35] Santti, U., Eskelinen, T., Rajahonka, M., Villman, R.,
Happonen, A. (2017), Effects of Business Model
Development Projects on Organizational Culture: A
Multiple Case Study of SMEs, Technology Innovation
Management Review, Vol. 7, Iss. 8, pp. 15-26, doi:
10.22215/timreview/1096
[36] Palacin, V., Gilbert, S., Orchard, S., Eaton, A., Ferrario,
M.A., Happonen, A. (2020). Drivers of Participation in
Digital Citizen Science: Case Studies on Järviwiki and
Safecast, Citizen Science: Theory and Practice, Vol. 5,
Iss. 1, Article: 22, pp. 1-20, doi: 10.5334/cstp.290
[37] Happonen, A., Santti, U., Auvinen, H., Räsänen, T.,
Eskelinen, T. (2020), Digital age business model
innovation for sustainability in University Industry
Collaboration Model, E3S Web of Conferences, Vol.
211, Article 04005, pp. 1-11, doi:
10.1051/e3sconf/202021104005
[38] Happonen, A., Stepanov, A. Hirvimäki, M., Manninen,
M., Dennisuk, W., Piili, H., Salminen, A. (2015), “Art-
technology collaboration and motivation sources in
technologically supported artwork buildup project”,
Physics Procedia, Vol. 78, pp. 407-414, doi:
10.1016/j.phpro.2015.11.0
[39] Sadovykh, A., Truscan, D., Pierini, P. et al. (2019), On
the Use of Hackathons to Enhance Collaboration in Large
Collaborative Projects - A Preliminary Case Study of the
MegaM@Rt2 EU Project, In DATE 2019 conference, pp.
498-503, doi: 10.23919/DATE.2019.8715247
[40] Uys, W. (2020), Hackathons as a Formal Teaching
Approach in Information Systems Capstone Courses, In
book: ICT Education, pp. 79-95, doi: 10.1007/978-3-
030-35629-3_6
