Content uploaded by Hanita Hassan
Author content
All content in this area was uploaded by Hanita Hassan on Jan 02, 2018
Content may be subject to copyright.
INCORPORATING HIGHER ORDER THINKING SKILL (HOTS) QUESTIONS 101
LSP International Journal, Vol. 4, Issue 1, 2017, 101–116
© Universiti Teknologi Malaysia
E-ISSN 2289–3199 / ISSN 0128–732X
Incorporating Higher Order Thinking Skill (HOTS) Questions in
ESL Classroom Contexts
Khairon Nisa Shafeei, Hanita Hassan*, Fauziah Ismail & Azian Abdul Aziz
Language Academy, Universiti Teknologi Malaysia, 81310 UTM Johor Bahru, Johor, Malaysia
ABSTRACT
Malaysia is preparing to face 21st century learning challenges in order to fulfill the global needs. As a result, Higher-Order
Thinking Skills (HOTS) have been introduced in the education system with the purpose to produce critical students who are
able to compete at the international arena. Thus, this study investigated the question types used by teachers of English as a
second language (ESL). Besides, this study also examined the challenges faced by the teachers in incorporating HOTS element
in their teaching. The data for this study were gathered by means of class observations, interviews and questionnaires. The data
collected by means of classroom observation and interviews were analysed thematically whilst the questionnaire data were
analysed using descriptive statistic method. The findings of this study reveal that ESL teachers favour display questions or
LOTS compared to referential questions, HOTS. This is due to lack of knowledge regarding HOTS questions and students'
English low proficiency level. This study, therefore, recommends that ESL teachers are in need of adequate training on HOTS
before they can successfully implement HOTS in language classrooms and further relevant studies should focus on the ways in
which HOTS can be incorporated in ESL classrooms.
Keywords: HOTS, ESL, question types, language classrooms
INTRODUCTION
Teachers use various question types as a means of measuring students’ understanding as well as to
ensure if the learning process of a particular subject has taken place (Kerry, 2002). Moreover, Kerry
(2002) also stated that types of questions used in the classroom play a significant role in the process
of teaching and learning as it affects students’ achievement and also level of engagement. It is
important for teachers to use the suitable question types in the classroom since students may give an
appropriate or intellectual response if the questions asked are well constructed (Jariah Mohd Jan
and Rosli Talif, 2005; Ribowo, 2006).
Effective questioning with incorporation of higher-order thinking skills (HOTS) is believed to be
more helpful to develop students’ level of critical thinking. Vijayaratnam (2012) indicates that
critical thinking by which students can relate and actively respond to real-world demands is what
we want. HOTS seems to be a crucial element in ESL classrooms because it stimulates students’
*Correspondence to: Hanita Hassan (email: hanita@utm.my)
102 Hanita Hassan et al.
critical thinking and according to Collins (1991), language arts achievement can be increased with the
use of teaching strategies along with the implementation of thinking competencies. Having said that,
Malaysia Education Development Plan (PPPM) 2013 to 2025 introduces the Higher Order Thinking
Skills (HOTS) to be integrated in the school syllabus with the purpose to generate knowledgeable and
critical students who are able to compete at the international arena. Nevertheless, there are on-going
debates among researchers whether or not Malaysian teachers and students are ready for HOTS.
This paper thus aims to discuss the question types used by ESL teachers and challenges faced in
incorporating HOTS in Malaysian contexts. The discussion will include the teachers' perceptions
towards the applicability of HOTS questions in ESL classrooms.
QUESTION TYPES USED IN SECOND LANGUAGE CLASSROOMS
Display and referential are two question types commonly used by ESL teachers. Display questions
can be defined as questions by which the answers are already known whilst referential questions are
known as open ended questions that focus on advices and opinion or questions that seek answer
naturally as a means of stimulating interactions (Thompson, 1997; Thornbury, 1996; Long & Sato,
1983; Brock, 1986). Referential questions demand more thought and require students to generate a
longer response and, most importantly, the teacher does not know the answer in advance (Brown,
2001). However, display questions are more dominant in ESL classrooms compared to referential
questions since display questions are a quicker tool to assess students' understanding of the text
contents (David, 2007; Boyd and Rubin, 2006).
Referential questions are used for more authentic communicative activities since they are likely to
be encountered in the ‘real world’ (David, 2007; Nunan, 1987; Long and Sato, 1983) and it is believed
that they could enhance students’ speaking skills. Besides enhancing students’ speaking skills,
referential questions, which are classified as HOTS, can also help develop students’ critical thinking,
although it is noted that HOTS questions are claimed to only suit high-achievers (Brown, 2001;
Zohar, et al., 2001). Following this, it is not easy for ESL teachers to adopt referential questions in
classroom, especially teachers in East Asia since students in this region typically prefer to listen
passively rather than voice out their ideas openly (Liu & Littlewood, 1997).
Teachers’ readiness to incorporate HOTS in their classrooms also plays a significant role. Given
the fact that there are teachers who are not aware of the importance of developing students’ critical
thinking in their lessons, and as a result, they avoid employing higher-level questions. Furthermore,
teachers find it difficult to incorporate HOTS in their classrooms and they are more comfortable with
teacher-centred approach (Nagappan, 2001; Sparapani, 1998). Tan and Mohammad Yusof Arshad
(2014), for instance, conducted a study on questioning technique in science classrooms, which used
problem based learning approach, and the findings indicated that there are different types of
questions used by teachers, which are higher-order, lower-order and eliciting ideas questions. They
stated that teachers tend to use low order questions which are 53% compared to higher-order
questions 20% and eliciting ideas questions 27% respectively. Thus, it clearly shows that Malaysian
teachers tend to use lower order thinking questions rather than higher-order questions.
INCORPORATING HIGHER ORDER THINKING SKILL (HOTS) QUESTIONS 103
The variety of teachers’ question types can result in active participation (Long & Sato 1983) and
therefore preparing good questions is one of the important factors that could lead students to generate
their own ideas, as well as, to help increase their confidence level and critical thinking (Effandi &
Zanaton, 2007). According to Mehan (1978), turn exchanges that occur between teachers and students
usually imply common acceptance and serious discussion. Sullivian and Lilburn (2004) listed three
features for good questions as follows:
1. Questions that help students require more information rather than just remembering and
reproducing them.
2. Questions that encourage students to learn by answering and expressing ideas while the
teachers will learn from the input delivered by the students.
3. Questions that may have more than two answers.
THE IMPLEMENTATION OF HOTS IN MALAYSIA
Higher-order thinking skills (HOTS) are known as the expanded of mind where the person must
analyze or interpret the answer or to manipulate the information since the correct answer is not
always gathered from the daily routine or knowledge (Onosko & Newman, 1994). The implementation
of HOTS in classroom environment is believed to enable students to think critically through active
learning. Teachers incorporating HOTS element in their teaching are expected to create interactive
questions or classroom activities, by which students can actively respond to the questions or generate
new ideas on the topic discussed. In HOTS classroom, students are no longer memorizing textbooks
but are expected to explore and to develop their critical thinking by means of the tasks given of which
they can relate to the real world situation (Krishnan and Yassin, 2009).
A project by the National Research Council, Committee on Research in Mathematics, Science, and
Technology Education (1987) which involved some American schools shows that the implementation
of HOTS in classrooms has an outstanding outcome towards the learning process. Students are found
more capable to use their long term memory compared to students who are exposed to the lower order
skill approach. The use of HOTS in classrooms will boost students’ confidence level and lead them to
be more critical, motivated and positive thinking in expressing and generating their ideas.
Even though Higher-Order Thinking Skills (HOTS) are the skills used in daily life, the skills are
difficult to be applied in learning and teaching process. Heong et al. (2012) conducted a study on the
needs analysis of Higher-Order Thinking Skills in generating ideas. The findings show that the
academic staff believe that technical students faced difficulties in solving their individual assignment
and also lack of knowledge in generating ideas. The reason for the incapabilities to solve problems and
generate ideas is lack of HOTS. It is therefore very crucial for the technical students to be equipped
with HOTS.
Nagappan (2001) investigated the perceptions of ESL and Malay language teachers towards
implementing HOTS in their classrooms. The findings show that the teachers are very much
interested in the teaching of Malay or English language but not in the application of the higher-order
thinking skills in their classroom. This obviously shows that some teachers are still lacking the
104 Hanita Hassan et al.
knowledge about HOTS, and this is identified as one of the challenges in implementing HOTS in
schools. Similar to Tan and Mohammad Yusof Arshad (2014), Habsah Hussin's (2006) study on HOTS
also found that teachers tend to use low level questions instead of high level question types. The result
showed that three teachers seemed to favour one answer from one student for each question before
moving on to the next question. This thus means that teachers do not expect other students to answer
the same question and this obviously discourages students’ participation.
Another study conducted on the implementation of HOTS in Malaysia was by Krishnan (2014).
The study was conducted to investigate teachers’ responses towards the idea of incorporating HOTS
in School Based Assessment (SBA). It was found that the teachers are aware of the importance of
HOTS in producing creative thinking society. The teachers stated that HOTS should be well prepared
and organized in order to encourage them to accept the implementation of HOTS since it can improve
students’ critical thinking. Unlike Krishnan (2014) who studied on the teachers’ perceptions,
Ganapathy and Kaur (2014) investigated the ESL students’ perception on the use of HOTS in English
language writing class. The findings show that HOTS motivate the students’ interest in writing
critically and promote students’ participation in the discussion with peers. Moreover, HOTS can be a
platform for students to compare, discuss and generate their own ideas. Therefore, the
implementation of HOTS could generate active learning and give positive impact to the teachers and
students.
Studies on teachers’ perceptions towards HOTS are still at its infancy, of which the previous
studies focused more on students’ feedback after the implementation of HOTS rather than teachers’
point of view while integrating HOTS in their teaching. Therefore, this study is interested to
investigate the question types used in ESL classrooms, teachers’ perceptions and the challenges faced
in the implementation of HOTS as a means of developing students’ critical thinking.
RESEARCH METHODOLOGY
The data collection methods for this study were class observations, interviews and questionnaires.
Class observations were conducted to identify the types of question asked and the challenges faced by
teachers in incorporating HOTS in ESL classrooms. Class observations on two lessons were video
recorded and the length for each lesson was thirty to thirty-five minutes. Both lessons were on reading
comprehension, using different reading texts based on the students’ levels of proficiency.
The questionnaires, on the other hand, were to measure teachers’ perceptions and beliefs in the
ability of HOTS questions in developing students’ critical thinking. There were three main sections:
Section A, Section B and Section C.
INCORPORATING HIGHER ORDER THINKING SKILL (HOTS) QUESTIONS 105
Table 1 Sections of Questionnaire
Section A
Demographic Information
Section B
Teachers’ Perceptions and Beliefs on HOTS Questions
Section C
Teachers’ Self-Reflection on Questioning Techniques used in Teaching and Learning
As shown in Table 1, teachers were asked about their demographic details such as gender, teaching
experience and their highest level of education in Section A. For Section B, teachers were asked on the
perceptions and beliefs towards implementing HOTS in ESL classrooms. The items in this section
were rated on a 3-point Likert-Scale (1= “disagree”, 2= “not sure” and 3= “agree”). Meanwhile,
Section C aims to identify teachers’ self-reflection on implementing HOTS in their lessons. The items
in this section were rated on a 3-point Likert-Scale (1= “seldom”, 2= “sometimes” and 3= “often”).
A total of twelve English secondary school teachers were the respondents, of whom eleven of them
(91.68 percent) were female while one (8.32 percent) male. In terms of classes taught, nine of them (75
percent) were currently teaching lower form meanwhile three of them (25 percent) were upper
secondary English school teachers. Eleven of them (91.68 percent) were bachelor degree holders
whereas only one (8.32 percent) was a master holder. However, only two teachers were selected for the
class observations and interviews. All twelve of them were involved in the implementation of HOTS
in the ESL programme.
Finally, the aim of the interviews was to triangulate the findings of the questionnaires.
Respondents were two English teachers whose lessons were recorded. They taught Form 3
intermediate and low level students. In this study, semi-structured interviews were carried out with
four open-ended questions. The questions asked were on teachers’ perceptions of HOTS, teachers’
techniques in employing HOTS question types and as well as the challenges faced in incorporating
HOTS as a means of cultivating students’ critical thinking.
The data from the class observations and interviews were analysed using thematic analysis and the
results were categorised into types of question favoured by teachers, teachers perceptions and
challenges of incorporating HOTS questions in ESL classrooms. In the thematic analysis carried out,
the questions posed by teachers were coded into two categories: HOTS and LOTS. The categories were
based on the six levels of Bloom’s Taxonomy, of which knowledge, comprehension and application
form linkages to LOTS since students are simply required to recall a single fact and do not involve
critical thinking. Meanwhile, HOTS questions include analysis, synthesis, and evaluation by which
students are requested to show their comprehension of the topic, situation or solution to a stated
problem. In this study, display questions are categorised as LOTS whilst referential questions are
HOTS.
The data from the questionnaires were analysed using descriptive statistical analysis and the
results were presented in the form of percentage. The challenges faced by teachers in implementing
HOTS were coded based on two factors, which are teachers’ perceptions of HOTS and students’
attitudes towards HOTS. In terms of teachers’ perceptions, three factors were identified which are
106 Hanita Hassan et al.
teachers’ knowledge about HOTS, teachers’ attitudes towards HOTS, and teachers’ skills in
employing HOTS questions in ESL classrooms.
QUESTION TYPES EMPLOYED BY ESL TEACHERS
The analysis done on the classroom observation data found that out of fifty-one questions posed by
teachers only two were referential questions (3.92%) whereas the other forty-nine were display
questions (96.08%). In addition, the findings from the questionnaires show that seven out of ten
teachers (70 percent) claimed they often use display questions which require short or one-word
answers rather than referential questions which demand longer answers related to their opinions.
According to Thornbury (1996), display questions refer to questions of which the teacher already
knows the answer and the kind that demands only a single word or short response. Referential
questions, on the other hand, demand deeper level cognitive processes and require the students to
produce a longer response. Additionally, the fact that referential questions are categorised as HOTS
and are more open in nature, the teachers may not know the answers in advance (Brown, 2001). We
can, therefore, infer that teachers prefer LOTS questions, or display question types, while HOTS
questions are not favoured in ESL classrooms.
Extract (T1) 1.1 illustrates some of display questions used by Teacher 1 (T1) who taught Form 3
intermediate class. The lesson was on reading comprehension, of which the teacher asked several
questions to test students’ understanding of the reading text.
Extract (T1)1.1: turns 8-13
8 T : okay, alright now... okay... aaa look at page six, look at page six. Okay, alright,
question C, read the travel brochure below then answer question A until I. Okay,
visit enchanting Kelantan, what’s the meaning of enchanting?
9 S : ermmm...
10 T : yes, what is the meaning of enchanting?... [Pause] yes, what is the meaning of
enchanting?...
11 S : ermm, cantik
12 T : in English?
13 S : Beautiful.
Based on Extract (T1)1.1: turns 8-13 above, the type of questions used by the teacher is
categorised as a display question as it clearly shows that the teacher already knew the answer. For
instance, when the teacher asked about the meaning of “enchanting” to her students, she expected
all of the students to know the meaning of the word since it can be found in the reading text. Based
on Bloom’s Taxonomy, this type of questions is labelled as knowledge as it involves the basic
cognitive process, such as, remembering and retrieving previously learned material. Learners are
only required to use the skill of remembering when the memory is used for producing definitions,
INCORPORATING HIGHER ORDER THINKING SKILL (HOTS) QUESTIONS 107
facts or retrieved materials (Anderson & Karthwohl, 2001). As mentioned earlier, most teachers
tend to use display questions rather than referential questions since teachers often assume that
display question is mainly a quicker way to assess whether the students have understood the
content of the text (David, 2007; Boyd and Rubin, 2006). Thus, HOTS question was not found in
this extract as the questions posed by T1 did not require students to integrate critical thinking in
their response and furthermore only short answer was expected from them.
Another example of display questions is found in Extract (T1) 1.2. Display question was used by
the teacher to ask the meaning of “compound”. Realistically, the teacher already knew the answer
and she expected the students to produce the same answer. In Extract (T1) 1.2, the teacher waited
for a while so that students could give the expected answer. For example, the teacher asked the
meaning of “compound” and the students said “surround” at the beginning. However, she waited
and guided the students until they could give the expected answer by adding suffixes “-ing” at the
end of the word. Giving more time for students to answer will lead students towards an active
participation (Nunan, 1990; 1991).
Extract (T1)1.2: turns 56-63
56 T : okay, very good. It’s a mosque... not mos queue. Okay... alright, now,
erm....[Pause]. Okay, erm... [Pause]. Compound? What’s the meaning of
compound?
57 S : (inaudible 11:36)
58 T : yes, in English please.
59 S : [cough]… errr...
60 T : compound?
61 S : surround, surround
62 T : surround…?
63 S : surrounding… surrounding
Based on the extract above it can be inferred that students gave full participation when waiting-
time was given. This was supported by Jiang's (2014) study, as it showed that teachers who did not
push their students to answer right away but rather gave the students more time to do some
research would make the students hold positive attitude about the lesson and help them understand
and remember the information easily.
Failing to sustain the interaction between teacher and students may lead to misunderstanding
between them, and thus, the objective of the lesson will not be achieved (Nunan, 1991). Extract
(T1) 1.3: turns 70-71 illustrates T1 used a true or false type of display questions to create interaction
with his students.
Extract (T1) 1.3: turns 70-71
70 T : [Pause] okay, alright… so… alright… by looking at the questions at page
seven, we have to look at page seven, okay… page seven [Pause]. Is on the travel
brochure state whether the following statements are true or false?
71 S : false.
108 Hanita Hassan et al.
It was found that there was a positive interaction between T1 and the students. All of the students
gave full participation in answering this type of questions. Kachur and Prendergast (1997) indicated
that the possibility for teachers to use display questions is to promote students dialogue. Thus, it can
be deduced that T1 used display questions to enable her students to give full participations during the
teaching and learning process. In other words, the display questions are actually used to monitor or
facilitate students to talk and to avoid them from keeping silent in language classrooms.
Some of the referential questions were identified in the observation transcriptions. Nevertheless,
the findings of this study revealed that HOTS questions are still underused in ESL classrooms. This is
because only two out of twenty-six questions were classified as HOTS questions and the rest were
categorised as LOTS. The examples of referential question found in the data are shown in the Extract
(T1) 1.4: turns 146-148 and Extract (T1) 1.5: turns 156-229.
Extract (T1) 1.4: turns 146-148
146 T : cave… Alright… [Pause] okay, why do you think the word amazing is used to
Describe Gunung Reng? [Pause] why…?
147 S : because it is an interesting place to visit
148 T : yes… okay, as a class can you answer it? Why do you think the word amazing is
used to describe Gunung Reng?
Extract (T1) 1.4: turns 146-148 illustrates an example of referential questions used by the teacher
which requires the students’ opinion about the word ‘amazing’ used to describe Gunung Reng. It was
a subjective question and the answers might vary. Students were required to justify their answer
through longer responses. As a result, this may give them opportunity to speak out in the target
language. Following Onosko and Newman (1994), who indicated that higher-order thinking skills are
known as the expanded of mind where the person must analyze or interpret the answer or to
manipulate the information since the correct answer is not always gathered from the daily routine or
knowledge, this question was thus categorised as HOTS. In addition, Anderson and Karthwohl (2001)
claim that HOTS questions require learners to break the concepts into different parts in order to
determine how those parts are related or interrelated to each other. Here, the student gave the answer
based on the knowledge he or she had and tried to relate why the word ‘amazing’ is used in describing
Gunung Reng.
Extract (T1) 1.5: turns 156-159 is another example of referential question which basically asked
about why the city of Kota Bharu was worth to explore by foot. Since there are many possible
answers to the questions which basically lie within the students’ opinion or general knowledge about
Kota Bharu, this type of questions resulted in active learning (Prince, 2004).
Extract (T1) 1.5: turns 156-159
156 T : lime…? Stone. Okay [Pause] okay, alright, why do you think the brochure says
That Kota Bharu is best explored on foot? Why..? Why? You have to
think..why..? This one you have one by one to answer, why? Why by foot?
INCORPORATING HIGHER ORDER THINKING SKILL (HOTS) QUESTIONS 109
157 S : because to get explored every eye catching places without worrying about the
transport and enjoy beautiful scenery [Pause]
158 T : Kota Bharu by foot. Why? You have to think. Why…? [Pause]
159 S : have some interesting place
In the interview session, T1 explained the importance of HOTS questions in developing students’
critical thinking skills. T1 also highlighted that regular use of HOTS questions can help students
relate those questions to their real life activities. Nevertheless, as mentioned earlier, only two out of
twenty six questions were categorised as HOTS questions. This signifies that the majority of the
questions used in class were display questions or LOTS.
The second respondent or T2 taught Form Three English class and the students were categorised
as low level students. The lesson recorded was on reading comprehension with a simpler reading text
used to match the students' level. The analysis of the classroom observation showed that there was no
HOTS question used. All of the questions posed by the teacher were categorised as display questions,
even though the teacher believes in the usefulness of HOTS questions for ESL students to develop
their critical thinking.
Several extracts are taken from the transcriptions to show the display questions used by T2. For
example, Extract (T2) 2.1 illustrates the display questions which are in the form of turn-taking of
initiation-response-evaluation (IRE).
Extract (T2) 2.1: turns 5-8
5. T: okay, thank you. [Pause] alright, okay, I will read once again… Awang Batil is
famous story-teller. What is the meaning of story- teller? [pause] yes… what is
the meaning of story-teller? Yes... someone raise up the hand. (Inaudible 01:54).
Okay, alright, what’s the… what is the meaning of story teller?
6. S: pencerita.
7. T: pencerita. The person who…?
8. S: err... tell about the story…
According to Mehan (1978), the three-turn sequence of initiation-response-evaluation (IRE), or
also known as Initiation-response-feedback (IRF), is the common sequential contexts of teacher-
student-teacher turn-taking found in ESL classrooms. In the “initiation” (I) phase, the teacher will
ask a question of which the students will respond to (R) and followed by feedback by the teacher (F).
Based on Extract (T2) 2.1, at the initiation phase (I) the teacher asked about the meaning of story-
teller and the students gave their answer (R) in Malay. The teacher then commented on the
inadequacy of the reply (F) and gave a clue to the answer “the person who…” which then enabled
students to explain the meaning of story teller. Given the fact that IRE sequences are an effective
means of monitoring and guiding students' learning (Christie, 1995; Mercer, 1992), it is thus suggested
that display questions should be used with low level students. Another example of a display question
used by T2 is illustrated in Extract (T2) 2.2.
110 Hanita Hassan et al.
Extract (T2) 2.2: turns 123-128
123. T : okay, very good. So… number eight the answer is…? C… wrapped…
present.. Okay…? [pause] okay, now, look at page number eight. Okay… look at
page number eight. (Inaudible 19:32). Okay, when we summarize a text, we can
categorise or classified with of these. There is no need to elaborate of this
example in your summary, okay..? Use your own word as far as possible and
check your grammar. Okay, now. These words or a phrase for each of the
following? Number one, okay, there is rambutan, langsat, orange, mango, apple.
What we called it? What we call it?
124. S :fruits
125. T :yes. Very good. the answer is…? Fruits. Okay. Okay, fruits. Okay, F, R, U, I, T,
S. fruits. Okay, what about nurse, doctor, teacher, postmen, architect?
126. S :(inaudible 20:43)... occupation.
127. T : yes, occupation. So, how to spell?
128. S: O, C, C, U, P, A, T, I, O, N, S
In Extract (T2) 2.2, the first question in turn 123 was a display question, which required students
to find an appropriate category for the words “rambutan, langsat, orange, mango and apple”. The
teacher expected the students to get the answer from the text since the question is identified as
‘knowledge’, that is the lowest category of cognitive process in Bloom’s Taxonomy. It is therefore
classified as a LOTS question.
In order to determine the frequency of display and referential questions used by the teachers, the
occurrences of both types of questions were calculated. Table 2 shows percentage of display and
referential questions found in the data. T1 used 24 display questions and 2 referential questions while
T2 eventually used only display questions in her classroom.
Table 2 Question Types Used by ESL Teachers
QUESTION TYPES
FREQUENCY
Intermediate
Low- level
T1
T2
Total
1. Display
Questions
24
25
49
(96.08%)
2. Referential Questions
2
0
2
(3.92%)
Overall Total
26
25
51
INCORPORATING HIGHER ORDER THINKING SKILL (HOTS) QUESTIONS 111
The results show that a total of 51 questions were asked by the teachers. Of the 51 questions, 49
questions (96.08 percent) were display questions while only 2 questions (3.92 percent) were referential
questions. Based on the findings, it can be claimed that the display questions were preferable
compared to referential questions, thus signifying that majority of the questions asked by the teachers
are categorised as a low-level type or LOTS. These results are supported by Nunan (1987) who stated
that a teacher usually asked predominantly display questions rather than referential questions in
class. The students’ low level of proficiency was the main reason for teachers to use display questions
but not referential questions.
The findings of the analysis done on T1 and T2 in classroom interactions revealed that most of the
questions were indeed LOTS where the answers required were in the form of multiple-choices (A, B, C,
and D), true or false, spelling, short phrases or one sentence answers. Table 3 shows the types of LOTS
questions employed by both teachers.
Table 3 Types of Display Questions (LOTS)
No.
Items
Teacher 1
Teacher 2
1.
Posing a series of questions
✓
✓
2.
Accepting one answer for each question
✓
✓
3.
Elaborating on students’ answers
✓
4.
Providing answers to own questions.
✓
✓
The results in Table 3 indicate that T1 and T2 had the tendency to pose a series of questions to
their students, however, T2 might have posed fewer questions compared to T1 because the students
are of different proficiency levels. T1 taught intermediate level Form 3 class whereas T2 taught low
level Form 3 class. In the classroom interaction observations, both teachers seemed to accept one
answer for each question before moving on to the next question. This shows that the teachers did not
expect or did not allow other students to respond to the same question. In conjunction to this,
Habsah Hussin (2006) in her research related to the higher-order thinking skills (HOTS) found that
teachers tend to use low level questions instead of high level questions. The teachers were observed to
be satisfied in accepting only one answer for each question before moving on to the next question
without allowing other students to answer the same question. By conducting this restrictive manner
of questioning, the students seemed to eventually become discouraged to participate in class.
In addition, the results from class interaction observations also revealed that ESL teachers have a
tendency to elaborate the answers given by students, in other words, to interject their own answers
into students’ responses without giving the students an opportunity to complete their answers. Extract
(T1) 2.4: turns 66, for instance, shows that T1 provided an answer to her question without giving
students an opportunity to respond:
112 Hanita Hassan et al.
Extract (T1) 2.4: turns 66
66 T : compound. Alright, now, guerillas… guerrillas… communist guerillas. What is
that? Guerillas? There are bunch of guys. There…lots of… lots of armies…or…
ermm…communist. Yes, that’s are mean like trying… Okay, communist some
kind of… the past…previous… okay, the past thing happened. Like when…
Malay… Malayan been attacked by communist. Communist who is trying…
okay? Bintang Tiga. You already know, right? (inaudible 12:46) who… want to
invaded Malaysia. That when… they are not. They can’t. Okay?[Pause] alright,
so… [Pause]. Legend, what is the meaning of legend?
The teacher formulated the question based on the reading text. Extract (T1) 2.4: turns 66 shows
that T1 gave the answer about the meaning of “guerrillas”. She elaborated the answer and did not
give the students an opportunity to share their knowledge about “guerrillas”. The findings of this
study confirm that ESL teachers tend to use display questions which test the students’ memory but
not their comprehension. This does not seem to fit well with the demands of present day teaching
which require the incorporation of deep level thinking skills. In other word, the questions in language
classrooms should also include referential types since referential questions emphasize on the meaning
so that students will have the chance to think critically, speak and participate more in the discussion.
CHALLENGES OF INCORPORATING HOTS QUESTIONS IN ESL CLASSROOMS
The findings of this study show that there are several challenges that teachers’ faced while employing
HOTS questions in ESL classrooms. The problems are clustered into two categories which include
teachers’ perceptions and students’ attitudes towards the implementation of HOTS in ESL
classrooms. As mentioned previously, teachers' perceptions include their knowledge on HOTS,
attitudes and skills in adopting HOTS questions in their lessons.
First and foremost this study found that lack of knowledge on how to implement HOTS is one of
the factors that contributes to the challenges in incorporating HOTS in ESL classrooms. In the
interview sessions with the teachers, T1 and T2 admitted that the problems came from teachers who
do not have sufficient knowledge about HOTS and apparently, both of them agreed that lack of
knowledge pertaining to HOTS is expected since HOTS is recently introduced to the education
system. In conjunction to that, Nagappan (2001) studied the teachers’ perceptions in terms of
readiness in conducting teaching and learning of HOTS from twenty-two selected schools. In the
study, teachers were asked about their knowledge regarding the curriculum for English and Malay
language teaching and also HOTS. The result shows that since some of the teachers lack the
knowledge of how to conduct a lesson on HOTS, they have the tendency to employ a teacher-centered
approach.
In addition, T1 claimed that so far questions asked in class tend to focus on the comprehension of the
subject matter rather than developing students' critical thinking. As a result, teachers normally only
INCORPORATING HIGHER ORDER THINKING SKILL (HOTS) QUESTIONS 113
accept one answer for each question before moving on to the next question since their ultimate aim is
to check students' understanding. However, it gives less opportunity for students to speak and give
their opinion. More interestingly, teachers also tend to provide answers to their own questions and
types of questions posed are basically depending on the students’ levels of proficiency. This thus
suggests that teachers need time to adapt to the new requirements.
More importantly, the results from the questionnaire clearly show that teachers are in need of
continuous training in the implementation of HOTS in L2 classrooms. In the interview sessions, both
interviewees,T1 andT2 stated that English teachers are in need of adequate training on HOTS before
they can successfully implement HOTS in their lessons. This might help them to gain knowledge as
well as experience when they have a chance to share views with others in employing HOTS questions
in their lessons and this, in turn, would benefit the teachers in the long run. In addition, teachers
could know which HOTS question is appropriate to use according to the students' level. In line with
this, Nickerson et al. (1985) claim that teachers need to be trained in order to be able to apply high
level thinking skills in class. It cannot be assumed that higher order thinking skills would emerge
automatically by maturation (Nickerson et al., 1985). In other words, teachers need to gain the
experience and have the knowledge about HOTS first before they can apply it in their classrooms.
Another striking problem that teachers encountered in implementing HOTS was students’
attitude. Based on the interview sessions, the teachers claimed that the students’ low level of
proficiency results in the great challenge for applying HOTS questions. Low achievers seem to have
difficulty to adapt to new changes. A report released by the Federal Inspectorate of Schools,
Malaysian Ministry of Education (2000) stated that students were still lacking in critical and creative
thinking skills. Most of them were unable to produce original and unique ideas, resulting in a hurdle
for teachers to apply HOTS in classrooms.
The low level students even have difficulties in responding to display questions. From the data,
two types of students were observed: 1) students who tend to use short answer; 2) students who did
not even understand the questions posed by the teachers.
It can, therefore, be deduced that the teachers used display questions to encourage students to
take part in the class interactions. In the interview session, T2 claimed that if she used referential
questions in class, the students might have problems in answering those questions and would
probably would refuse to answer the questions at all. Analysis from the questionnaire also show
similar results, by which 70 percent of the teachers stated that their students often responded with
short answers and less number of students participated when teachers posed HOTS questions.
Likewise, Boyd and Rubin (2006) claimed that text based talk is presumed to encourage lower levels
of comprehension and entail display questions, which elicit single-word answers rather than extended
discussion.
CONCLUSIONS AND RECOMMENDATIONS
The findings of this study found that ESL teachers tend to use display questions rather than
referential questions in their classrooms. Even though there were some referential questions identified
from the data, the frequency for the referential question used was very small. One of the reasons is
114 Hanita Hassan et al.
that most of the students gave positive feedbacks towards display questions compared to referential
questions as this type of questions are much easier to answer and do not require critical thinking. This
concludes that the use HOTS questions is still limited and not favoured in ESL classroom contexts.
Even though there is a limited use of HOTS questions, majority of the respondents give positive
feedback towards the implementation of HOTS in ESL classrooms. Most teachers are aware of the
importance of HOTS, however, they face some challenges in applying HOTS due to lack of
knowledge, teachers attitudes and students' low proficiency levels.
We identified several implications of this study, for example, to incorporate HOTS questions in
ESL classrooms, teachers should first be aware of their students’ level of proficiency since students'
proficiency levels determine the suitable types of HOTS questions to be used. The HOTS questions
should correspond to the students’ real life situations and thus, this can generate students’ interests as
well as motivation. Besides that, teachers also need to prepare themselves in terms of their knowledge,
pedagogical skills and attitude in implementing HOTS questions and this can be achieved through
attending HOTS training courses (Stiggins, Griswold, and Wikelund, 1989).
The future studies on HOTS should look into students’ perceptions towards HOTS besides
teachers’ perceptions. It would be appropriate if the future researchers investigate students’
perceptions as well to get more reliable data. In addition, studies on how the HOTS questions can be
successfully incorporated in ESL classrooms are highly needed.
ACKNOWLEDGEMENT
We would like to acknowledge and extend our gratitude to the Ministry of Higher Education
(MOHE), Malaysia and Research Management Centre, Universiti Teknologi Malaysia for the research
fund under Fundamental Research Grant Scheme R.J130000.7801.4F510.
REFERENCES
Anderson, L. W. & Krathwohl, D. R. (Eds.). 2001. A Taxonomy for Learning, Teaching and Assessing:
a Revision of Bloom's Taxonomy of Educational Objectives: Complete Edition. New York:
Longman.
Boyd, M. & Rubin, D. 2006. How Contingent Questioning Promotes Extended Student Talk: A
Function of Display Questions. Journal of Literacy Research. 38(2): 141-169.
Brock, C. A. 1986. The Effects of Referential Questions on ESL Classroom Discourse. TESOL
Quarterly. 20(1): 47-59.
Brown, H. D. 2001. Teaching by Principles: An Interactive Approach to Language Pedagogy. 2nd ed.
New York: Longman.
Christie, F. 1995. Pedagogic Discourse in the Primary School. Linguistics and Education. 7(3): 221-
242.
Collins, C. 1991. Reading Instruction that Increases Thinking Abilities. Journal of Reading. 34(7):
510-516
INCORPORATING HIGHER ORDER THINKING SKILL (HOTS) QUESTIONS 115
David, O. F. 2007. Teachers' Questioning Behavior and ESL Classroom Interaction Pattern.
Humanity and Social Science. 2(2): 127-131.
Effandi, Z. & Zanaton, I. 2007. Promoting Cooperative Learning in Science and Mathematics
Education: A Malaysia Perspective. Eurasia Journal of Mathematics, Science & Technology
Education. 3(1): 35-39.
Ganapathy, M. & Kaur, S. 2014. ESL Students’ Perceptions of the use of Higher Order Thinking
Skills in English Language Writing. Advance in Language and Literary Studies. 5(5): 80-87.
Habsah Hussin. 2006. Dimensions of Questioning: A Qualitative Study of Current Classroom Practice
in Malaysia. Teaching English as a Second or Foreign Language. 10(2): 1-18.
Heong, Y. M., Yunos, J. M., Othman, W., Hassan, R., Kiong, T. T. & Mohamad, M. M. 2012. The
Needs Analysis of Learning Higher Order Thinking Skills for Generating Ideas. Procedia-Social
and Behavioral Sciences. 59: 197-203.
Jariah Mohd Jan. & Rosli Talif. 2005. Questioning Strategies and the Construction of Context in
Classroom Talk. The English Teacher. XXXIV: 76-89.
Jiang, Y. 2014. Exploring Teacher Questioning as a Formative Assessment Strategy. RELC Journal,
45(3): 287-304.
Kachur, R. & Prendergast, C. 1997. A Closer Look at Authentic Interaction: Profiles of Teacher–
Student Talk in Two Classrooms. In M. Nystrand (Ed.). Opening Dialogue: Understanding the
Dynamics of Language and Learning in the English Classroom. New York: Teachers College
Press. 75-88.
Kerry, T. 2002. Explaining and Questioning. Cheltenham, UK: Nelson Thornes.
Krishnan, M. & Yassin, R. M. 2009. Problem based Learning in Engineering Education at Malaysian
Polytechnics: A Proposal. 2009 International Conference on Engineering Education, ICEED2009
- Embracing New Challenges in Engineering Education. 122-124.
Krishnan, B. 2014. The Acceptance and Problems Faced by Teachers in Conducting Higher Order
Thinking Skills. Doctoral dissertation, Universiti Teknologi Malaysia, Faculty of Education.
Liu, N. F. & Littlewood, W. 1997. Why Do Many Students Appear Reluctant to Participate in
Classroom Learning Discourse? System. 25(3): 371-384.
Long, M. & C. Sato. 1983. Foreigner Talk Discourse: Forms and Functions of Teachers’ Questions.
Classroom-Oriented Research on Second Language Acquisition. H. Selinger & M. Long, (eds.).
Rowley, MA: Newbury House. 268-285.
Malaysia Ministry of Education. 2000. Dapatan Nazir Tentang Kualiti Pengajaran & Pembelajaran
Dan Pengurusan Sekolah.
Malaysia Ministry of Education. 2012. Malaysia Education Blueprint 2013-2025: Preliminary Report.
Mehan, H. 1978. Structuring School Structure. Harvard Educational Review. 48(1): 32-64.
Mercer, N. 1992. Task for Teaching and Learning. K. Norman (Ed.). Thinking Voices: The Work of the
National Oracy Project. London: Hodder & Stouehton. 215-223.
Nagappan, R. 2001. Language Teaching and the Enhancement of Higher-Order Thinking Skills. W.
A. Renandaya & N. R. Sunga (Eds.). Language Curriculum and Instructions in Multicultural
Societies. Anthology Series (42). Singapore: Seameo Regional Language Centre. 190-223.
National Research Council, Committee on Research in Mathematics, Science, and Technology
Education. 1987. Education and Learning to Think. United States of America: National
Academy Press.
116 Hanita Hassan et al.
Nunan, D. 1987. Communicative Language Teaching: Making It Work. ELT Journal. 41/2: 136-45.
Nunan, D. 1990. The Question Teachers Ask. JALT Journal. 12(2): 187-201.
Nunan, D. 1991. Language Teaching Methodology. Hemel Hemstead: Prentice Hall.
Onosko, J. J. & Newmann, F. M. 1994. Creating More Thoughtful Learning Environments. J.
N.Mangieri & C. C. Blocks (Eds.). Creating Powerful Thinking in Teachers and Students: Diverse
Perspectives. Fort Worth: Harcourt Brace College Publishers. 27-49.
Ribowo, B. 2006. Upaya Meningkatkan Hasil Belajar Siswa Kelas II A SMP Negeri 2 Banjarharjo
Brebes dalam Pokok Bahasan Segiempat Melalui Model Pembelajaran Tutor Sebaya dalam
Kelompok Kecil Tahun Pelajaran 2005/2006. Doctoral dissertation, Universitas Negeri
Semarang.
Sparapani, E. F. 1998. Encouraging Thinking in High School and Middle School: Constraints and
Possibilities. The Clearing House. 71(5): 274-276.
Sullivan, P. & Lilburn, P. 2004. Open-ended Maths Activities: Using ‘Good’ Questions to Enhance
Learning in Mathematics. 2nd ed. South Melbourne, Vic.: Oxford University Press.
Tan, Y. P & Mohammad Yusof Arshad. 2014. Teacher and Student Questions: A Case Study in
Malaysian Secondary School Problem-Based Learning. Canadian Center of Science and
Education. 10(4): 174-182.
Thompson, G. 1997. Training Teachers to Ask Questions. ELT Journal. 51(2): 99-104.
Thornbury, S. 1996. Teachers Research Teacher Talk. ELT Journal. 50(4): 279-289.
Vijayaratnam, P. 2012. Developing Higher Order Thinking Skills and Team Commitment via Group
Problem Solving: A Bridge to the Real World. Procedia-Social and Behavioral Sciences. 66: 53-
63.
Zohar, A., Degani, A., & Vaaknin, E. 2001. Teachers’ Beliefs About Low-Achieving Students and
Higher Order Thinking. Teaching and Teacher Education. 17(4):469-485.
