Available via license: CC BY-NC-SA
Content may be subject to copyright.
Communications in Information Literacy
/,4-& :224& 13*$,&
Teaching and Learning Information Synthesis: An Intervention
and Rubric Based Assessment
Kacy Lundstrom
Utath State University+"$8,4.%231/-424&%4
Anne Diekema
Utah State University"..&%*&+&-"424&%4
Heather Leary
University of Colorado Boulder)&"3)&1,&"18$/,/1"%/&%4
Sheri Haderlie
Utah State University2)&1*)"%&1,*&424&%4
&342+./6)/6"$$&223/3)*2%/$4-&.3#&.&<328/4
/,,/63)*2".%"%%*3*/.",6/1+2"3 )=00%72$)/,"1,*#1"180%7&%4$/--*.'/,*3
"13/'3)& .'/1-"3*/.*3&1"$8/--/.2
;*2&2&"1$)13*$,&*2#1/4()33/8/4'/1'1&&".%/0&."$$&223)"2#&&."$$&03&%'/1*.$,42*/.*./--4.*$"3*/.2*..'/1-"3*/.*3&1"$8#8".
"43)/1*9&%"%-*.*231"3/1/'!$)/,"1/1-/1&*.'/1-"3*/.0,&"2&$/.3"$3 0%72$)/,"10%7&%4
&$/--&.%&%*3"3*/.
4.%231/-"$8*&+&-"..&&"18&"3)&1".%"%&1,*&)&1*&"$)*.(".%&"1.*.(.'/1-"3*/.8.3)&2*2.
.3&15&.3*/.".%4#1*$"2&%22&22-&.3 Communications in Information Literacy /,2213*$,&
5"*,"#,&"3 )=00%72$)/,"1,*#1"180%7&%4$/--*.'/,*35/,*22
TEACHING AND LEARNING INFORMATION
SYNTHESIS
An intervention and rubric based assessment
Kacy Lundstrom
Utah State University
Anne R. Diekema
Utah State University
Heather Leary
University of Colorado Boulder
Sheri Haderlie
Utah State University
Wendy Holliday
Northern Arizona University
The purpose of this research was to determine
how information synthesis skills can be taught
effectively, and to discover how the level of
synthesis in student writing can be effectively
measured. The intervention was an information
synthesis lesson that broke down the synthesis
process into sequenced tasks. Researchers
created a rubric which they used to assess
students’ levels of information synthesis
demonstrated in their final research essays. A
form of counting analysis was also created to
see if other methods could help in measuring
synthesis.
Findings from the rubric analysis revealed that
students appear to benefit from the synthesis
lesson. The level of synthesis, however,
remains low overall. In addition, the study
showed that the different measures of synthesis
established were able to identify different
levels of information integration. Discovering
effective ways to measure and teach synthesis
continues to be essential in helping students
become information literate.
60
[ARTICLE]
Volume 9, Issue 1, 2015
Lundstrom et al.: Teaching and Learning Information Synthesis: An Intervention and
Published by PDXScholar, 2017
INTRODUCTION
Perhaps the most essential, and certainly
one of the most complex research skills, is
the ability to synthesize information. One
researcher, J.D. Johnson (2009) writes: “…
the ability for people to assimilate
information they find into coherent personal
strategies is perhaps the critical modern
survival skill” (p. 601). Information
synthesis is the process of analyzing and
evaluating information from various
sources, making connections between the
information found, and combining the
recently acquired information with prior
knowledge to create something new.
Information synthesis strategies are essential
skills. Without them, we cannot derive new
knowledge from these large amounts of data
(Larsen, Wactlar & Friedlander, 2003;
National Science Board, 2005). Effective
information synthesis is also vital in
developing effective writing and
communication skills to share new
knowledge. Coherent information synthesis
is, therefore, required to productively
participate in and contribute to our
information-rich society. Yet college
students have difficulty analyzing and
synthesizing different pieces of information
(Howard, Serviss, & Rodrigue, 2010).
The research questions explored in this
study ask whether information synthesis
skills can be taught effectively by
scaffolding this complex cognitive task, and
how the level of synthesis in student writing
can be effectively measured. The study
described here investigates an information
synthesis lesson given to students in a
university English writing class. The lesson
broke down the synthesis process into
several stages requiring students to go
through information synthesis in discrete
steps by means of individual reading and
group discussion. At the end of the lesson
students were asked to write a synthesized
paragraph. The authors also collected final
research papers in the class. Both writing
products were analyzed for evidence of
synthesis to evaluate the impact of the
synthesis lesson as well as to establish
different metrics for measuring synthesis.
Findings from the study revealed that
students appear to benefit from the synthesis
lesson. There were more instances of
information synthesis in the final papers of
students who received the lesson. The level
of synthesis however, remained low overall.
In addition, studies measuring synthesis
identified different levels of information
integration. The synthesis rubric used in this
study reaffirms that synthesis, and the
assessment of it, includes numerous skills
and competencies. The implications of these
findings suggest that teaching synthesis
through scaffolding this process requires
more than a single lesson and should
perhaps be provided early in the semester.
Rubrics and additional metrics that identify
synthesis can be used to communicate to
students about certain features of
synthesized papers and can help instructors
and librarians to more accurately assess
student work and provide them meaningful
feedback for improvement.
PROBLEM STATEMENT
The literature on information seeking
behavior shows that students have
superficial information seeking and research
skills (Asher, Duke, & Green, 2010;
Fitzgerald, 2004; Head and Eisenberg,
2009; Kolowich, 2010). In one study only
Lundstrom et al, Teaching & Learning Info Synthesis Communications in Information Literacy 9(1), 2015
61
[ARTICLE]
Communications in Information Literacy, Vol. 9, Iss. 1 [2017], Art. 4
http://pdxscholar.library.pdx.edu/comminfolit/vol9/iss1/4
50% of student participants were able to
successfully synthesize information from
multiple sources (Mateos & Sole, 2009).
Information synthesis is a key skill for
participants in our knowledge society and
requires complex processing (Fitzgerald,
2004; Goldman, 2004). Yet information
literacy instruction and practice tend to
favor easily-defined skills that often only
emphasize the search component of the
research process, leaving out higher order
processes like information synthesis (Lloyd,
2007; Montiel-Overall, 2007; Simmons,
2005; Tuominen, Savolainen, & Talja,
2005; Webber & Johnston, 2000). Similarly,
in the writing classroom, teachers are
largely unfamiliar with how to teach
synthesis sometimes implying it is a linear
process (McGinley, 1992), leading Mateos
and Sole (2009) to call for a “unique,
careful teaching approach” (p. 448). In
response, this study seeks to address the
question of how to teach students to
effectively synthesize information from
multiple sources, and how to effectively
assess and identify synthesis when it occurs
in student work.
LITERATURE REVIEW
Information synthesis appears in the
literature under several different guises.
Commonly used terminology to describe the
process of analyzing and evaluating
information from various sources is multiple
document (or source) comprehension
(Goldman, Braasch, Wiley, Graesser, &
Brodowinska, 2012; Goldman &
Scardamalia, 2013), multiple document
processing (Goldman & Scardamalia, 2013),
transliteracy (Andretta, 2009; Thomas et al.,
2008), intertextuality (Goldman, 2004;
Stadtler & Bromme, 2013), writing from
sources (Howard, Serviss, & Rodrigue,
2010), and information synthesis (Blake &
Pratt, 2002; Goldschmidt, 1986). Relevant
literature to this study can be found in
library and information science (information
literacy and information problem solving
models), education (cognition and literacy
instruction), and composition and rhetoric
(writing). Each field approaches the subject
from a different angle. Different research
and resulting instructional approaches of the
various fields are discussed below.
Information Problem Solving Models
Information problem solving models, also
known as information literacy models,
mostly serve as scaffolds for teaching the
research process or as frameworks when
studying the same process. Information
synthesis appears in all of the most well-
known models. The The Big 6 model is an
information problem solving model
developed by Eisenberg and Berkowitz, and
is used widely in K-12 schools. This model
includes synthesis as step five in their six-
step stages, which also includes task
definition, information seeking strategies,
location and access, use of information, and
evaluation (Eisenberg & Berkowitz, 1990;
Lowe & Eisenberg, 2005). This particular
step includes organizing from multiple
sources and then presenting the information.
Here students are directed to read and then
write from their notes from previous steps
and to reflect on how best to present this
information. In Stripling's six-phase Model
of Inquiry, synthesis is contained in the
construct phase (Stripling, 2010). This is
where the bridge is built from previous
knowledge to draw new conclusions, where
conflicting information is confronted,
conclusions are drawn, and evidence-based
opinions are formed. In the Information
Lundstrom et al, Teaching & Learning Info Synthesis Communications in Information Literacy 9(1), 2015
62
[ARTICLE]
Lundstrom et al.: Teaching and Learning Information Synthesis: An Intervention and
Published by PDXScholar, 2017
Search Process (ISP), Kuhlthau does not use
the label of synthesis, but does include the
process itself in the formulation stage
(Kuhlthau, 1991; Kuhlthau, Heinström, &
Todd, 2008). The formulation stage is the
fourth of six stages, where the learner forms
a focus from all the information
encountered, identifying and selecting ideas
from multiple sources to form a focused
perspective.
Information synthesis, a higher form
of educational thinking
Information synthesis most commonly
appears in the education literature as a level
in the original Bloom’s Taxonomy of
Learning Domains (Bloom, 1956). This
Taxonomy is a classification for
understanding student learning and to
promote higher forms of educational
thinking. The Taxonomy is often depicted
as a pyramid with the higher forms of
thinking at the top. For understanding the
cognitive domain of learning, the Taxonomy
builds upon steps beginning with factual
knowledge and moving to comprehension,
application, analysis, synthesis, and
evaluation. Synthesis, ranked second from
the top in the original Taxonomy, is
considered one of the most important goals
in the field of education. Here, synthesis is
defined as the building of structures or
patterns from a variety of elements with
emphasis on creating some new meaning or
a new structure from the elements. Some of
the keywords involved in synthesis include:
combine, create, design, and summarize. In
Krathwohl’s (2002) revision of Bloom’s
Taxonomy, the original categories were
renamed and their definitions revised to
represent more active thinking. Synthesis
was renamed to “create” and changed places
with “evaluation” as the top category in the
domain. The create category is defined as
putting together elements to make a whole,
including the elements of generating,
planning, and producing.
The role of synthesis in text
comprehension
Information synthesis can be seen in the
area of text comprehension, specifically in
multiple-source comprehension (also known
as multiple document processing, or
intertextuality). Historically, text
comprehension research involved single-
document comprehension; this was not
extended to multiple texts until the 1990s
when Wineburg (1991) studied how novices
and experts reasoned about a historical
event using multiple documents. Using
think-aloud protocols Wineburg identified
the strategies people used to come to a
conclusion. More researchers followed (see
Stadtler & Bromme, 2013 for details),
resulting in a better understanding of the
various strategies employed when
processing multiple documents (e.g.
Perfetti, Rouet, & Britt, 1999). According to
Goldman and Scardamalia (2013), to be
successful at synthesizing information from
multiple documents, students need to be
taught content knowledge, source expertise,
and an understanding of how knowledge is
created in the field of study. Once these are
in place students can evaluate information,
integrate it into existing belief structures,
and create new knowledge. Both Jucks and
Paus (2013) and Goldman and Scardamalia
(2013) note the social aspect of creating
meaning, and they emphasize the use of
discussion when teaching multiple-
document processing in general and the
resolution of conflicting information
between documents. Based on this research,
the current study incorporates group
Lundstrom et al, Teaching & Learning Info Synthesis Communications in Information Literacy 9(1), 2015
63
[ARTICLE]
Communications in Information Literacy, Vol. 9, Iss. 1 [2017], Art. 4
http://pdxscholar.library.pdx.edu/comminfolit/vol9/iss1/4
discussion in the intervention, alternated by
individual reading and reflection.
Synthesis in the writing classroom
The rhetoric and composition literature
addresses how students learn to synthesize
multiple texts. Synthesis is complex in
nature; therefore, the reading processes, the
writing processes, and writing from multiple
sources are all relevant to this discussion.
This literature also considers synthesis to be
a cognitively demanding task (Mateos,
Martin, Villalon & Luna, 2008), which
requires multiple activities such as
organization, comprehension, problem
detection, and problem solving (Bråten, I.,
Strømsø, 2003). Similarly, Nelson and
Hayes (1988) noted that in order to write
from multiple sources, students had to
“coordinate a number of supporting
activities.” Flower, et al. (1990) determined
that synthesis is a risky endeavor, where the
reader’s experience and knowledge, the text,
and “reality itself may resist synthesis” (p.
50). Not surprisingly, only 50% of those
high school and university students who
were studied could successfully synthesize
(Mateos & Sole, 2009). Torraco (2005)
views synthesis as a creative activity “that
produces a new model, conceptual
framework, or other unique conception
informed by the author’s intimate
knowledge of the topic” (p. 362). The same
author describes four forms of synthesis,
including a research agenda, a taxonomy or
classification construct, alternative models
or conceptual frameworks, and meta-
theories.
Nancy Spivey (1989), a major contributor to
research in this area, elaborates on the major
components of discourse synthesis,
including the ability to “select, organize and
connect content from sources texts as they
compose their own new texts” (p. 9). Like
Wineburg (1991), both McGinley (1992)
and Spivey (1984) note that much variation
existed in students’ writing processes with
proficient and non-proficient college readers
making different decisions in the ways they
chose to make connections between texts.
This literature also focuses on how to
effectively teach the synthesis process.
Mateos and Sole (2009) found that very few
teachers knew how to help students go
beyond connecting main ideas between
different sources, while McGinley (1992),
looking at the connection between writing
and thinking processes, concluded that
teachers should avoid implying that writing
from multiple sources is wholly linear.
Following a collaborative approach similar
to the current study, Fluellen (2011) paired
students together to read aloud and map
concepts. McGregor (2011) used coded,
graphical representations of student work to
start conversations about how students use
sources. Another solution requires
instructional technology to teach synthesis
at younger stages, such as TurboCite or
TurboWrite (Tooley, 2005).
Measuring information synthesis
The usefulness of using rubrics to help
measure information literacy skills has been
well documented by Oakleaf and others.
Oakleaf comments on the ability of a rubric
to “capture useable data about information-
seeking behavior,” and on the value of the
rubric development process itself (2007, p.
28).
While no comprehensive rubric exclusively
evaluating information synthesis was found,
Lundstrom et al, Teaching & Learning Info Synthesis Communications in Information Literacy 9(1), 2015
64
[ARTICLE]
Lundstrom et al.: Teaching and Learning Information Synthesis: An Intervention and
Published by PDXScholar, 2017
numerous rubrics have been created that
evaluate information literacy, and they
typically include elements of synthesis.
Commonly found aspects of synthesis are as
follows: establishing associations between
texts, recognizing patterns among
information (similarities, differences,
unique instances), organizing information to
express these relationships and patterns by
using transitional sentences, and other
explicit or implicit markers. Relevant
rubrics include the Inquiry and Analysis
VALUE Rubric (Association of American
Colleges and Universities [AACU], 2010),
Northern Arizona University’s (NAU)
Synthesis Essay Rubric, Rubric Assessment
of Information Literacy Skills (RAILS)
Using Information to Accomplish a Purpose
Rubric, the Evergreen Synthesis Paper
Rubric (Ford), and the General Education
Assessment Rubrics (Klassen, 2014). We
determined that combining and rethinking
pieces of these current rubrics would best
suit the task of determining whether or not
synthesis was present in student papers.
The rubrics listed above and an in-depth
analysis of numerous student papers
informed the creation of the information
synthesis rubric (see Appendix A) used in
this study. The VALUE rubric was
beneficial as it was developed by teams of
faculty at colleges and universities across
the United States. The aim of the VALUE
project, which resulted in the creation of 16
different rubrics, was to have a national
framework to support common dialog and
understanding in specific areas for
undergraduate level work (AACU, 2010).
The Inquiry Analysis rubric heavily
informed a category (subscale D) in our
rubric which focused on source
organization. The Synthesis Essay Rubric
was a collaborative effort between NAU’s
eLearning Center and faculty to design
effective instruction and assessment. The
Synthesis Essay Rubric was especially
important in creating subscale C in the
present study: identifying conversations.
The Using Information to Accomplish a
Purpose Rubric was created for a library
instruction teaching workshop focused on
assessment and deposited in the RAILS
repository, a funded research project
investigating the use of analytical rubrics in
the assessment of information literacy.
Terry Ford, from Evergreen State College,
created the Synthesis Paper Rubric. Ford’s
rubric was helpful in providing vocabulary
to describe different skill levels for the
present study: emerging, developing, and
proficient. Ford’s rubric was also used
extensively to create subscale E: analyzing
sources to create something new. Klassen’s
General Education Assessment Rubrics
(2014) helped to distinguish levels of
progression as students become more
proficient in developing a range of skills
essential to general education. The rubric
adapted two of the seven categories in the
Synthesis Rubric, including information
literacy skills and synthesis and critical
thinking skill patterns. The researchers
relied on these five rubrics to develop a
rubric which more adequately provided a
guide for identifying synthesis in student
work.
RESEARCH DESIGN AND
METHODS
The research team included two faculty
librarians with a focus on library instruction,
and two teaching faculty from the
Instructional Technology & Learning
Sciences Department, all with a vested
Lundstrom et al, Teaching & Learning Info Synthesis Communications in Information Literacy 9(1), 2015
65
[ARTICLE]
Communications in Information Literacy, Vol. 9, Iss. 1 [2017], Art. 4
http://pdxscholar.library.pdx.edu/comminfolit/vol9/iss1/4
interest in helping students improve their
synthesis skills. The researchers developed
an information synthesis lesson, which was
implemented in four sections of an English
2010 class. English 2010, Intermediate
Writing: Research Writing in a Persuasive
Mode, is a required second-year
composition class. The focus of the class,
according to the course description, is the
“writing of reasoned academic argument
supported with appropriately documented
sources. [It] focuses on library and Internet
research, evaluating and citing sources, oral
presentations based on research, and
collaboration.” Writing products were
collected at the end of the intervention in
addition to students’ final papers for the
course. These products were analyzed using
the synthesis rubric that was created and
adapted from the previously mentioned
rubrics.
Participants
The 87 study participants were enrolled in
four Spring 2013 sections of English 2010,
all taught by the same instructor. The
enrollments for the four sections consisted
of 21, 22, 23, and 21 students respectively.
While biographical data on individual
students was not collected, the total
participant pool was 32 female students and
55 male students. There were five freshmen,
29 sophomores, 34 juniors, and 19 seniors.
For the paper analysis, nine papers were
randomly selected from each section.
The control group participants were enrolled
in four Fall 2011 sections of English 2010,
all taught by the same instructor. There were
89 total control group students, of whom 44
were female and 45 were male. There were
six freshmen, 26 sophomores, 37 juniors,
and 20 seniors.
Measures
To measure the extent to which information
synthesis took place in the written products
(synthesis paragraphs and final papers),
researchers in this study developed an
information synthesis rubric based on an in-
depth analysis of student papers and a series
of information literacy rubrics that included
aspects of synthesis. The rubric consisted of
five categories: A. Source variety, B. Uses
information from sources effectively, C.
Identifies conversations among information
from different sources, D. Organizes
sources overall in a meaningful, purposeful
way, and E. Analyzes sources to create
something new or draw conclusions and
make generalizations. These categories were
scored as unacceptable (0), needs
improvement (1), developing (2), advanced/
mastery (3). The rubric was revised through
three iterations in which researchers applied
it to a total of ten student papers from the
same course in a previous semester. This
initial assessment process ensured
consistency between raters in measuring
students’ level of proficiency in
synthesizing information, and resulted in the
final rubric version in Appendix A.
Procedures
The information synthesis lesson was
implemented in four sections of ENGL
2010. The same researchers taught all four
sections. The lesson (see Appendix B)
lasted approximately 75 minutes and broke
the synthesis process down into sequenced
tasks. The lesson was based on an
information synthesis workshop developed
by Johnson at Arizona State University
West (2003). Students were placed in
groups of three. Using PowerPoint, the
librarians provided instructions and
discussed the characteristics of synthesis.
Lundstrom et al, Teaching & Learning Info Synthesis Communications in Information Literacy 9(1), 2015
66
[ARTICLE]
Lundstrom et al.: Teaching and Learning Information Synthesis: An Intervention and
Published by PDXScholar, 2017
After basic definitions were established, the
guided practice began. Each step of the
guided practice had clear time delineations
with corresponding instructions on a
PowerPoint slide. Students were instructed
to read two articles. Each student read one
unique article and one article in common
with the other members of his or her group.
Students each highlighted the main points or
areas about which they had questions. Then
students wrote the five main points of their
assigned articles on post-it notes, shared
their notes with group members, and
worked together to group main points by
topic. Once students established general
categories, they regrouped their post-it
clusters and titled each one. At the end of
the lesson, students were instructed to
individually write a paragraph that
synthesized one of their group’s clusters.
Two researchers observed the lessons to
record students’ reactions, paying special
attention to student questions and their
difficulties with particular steps in the
process. After each lesson, the researchers
asked for student and instructor feedback
through debriefing and a short online
survey. They collected students’ clusters of
concepts derived from in-class readings for
later analysis, and at the end of the semester,
they collected students’ final papers as well.
RESULTS
The researchers scored all student papers
using the rubric and tested their inter-rater
reliability. They then measured whether the
information synthesis intervention improved
student synthesis skills, based on student
papers and the synthesized paragraphs.
Finally, the researchers established whether
counting visible markers of synthesis could
predict the level of synthesis in student
papers scored by the rubric.
The information synthesis rubric
To test the information synthesis rubric for
reliability, the researchers scored 72 final
student papers (36 from the control group
and 36 from the treatment group). Each
paper was rated by two raters individually
and then discussed to form a consensus. To
test the reliability of the rubric, inter-rater
reliability was calculated using Cronbach’s
alpha. Raters had acceptable reliability
overall (α = .72); the breakdown was α
= .73 on control papers and α = .71 on
treatment papers. Cronbach values between
0.70 and 0.95 are considered to be
acceptable (Tavakol & Dennick, 2011).
The impact of the information
synthesis lesson
The researchers calculated the difference
between the control and treatment group
papers for each subscale. To determine if
there was a significant difference between
the calculated scores of the control and
treatment papers, an analysis of variance
was calculated for each of the five subscales
of the rubric. Subscales A, B, D, and E had
no significant differences. A main effect F
(1, 70)=7.36, p<.01 was reported for
Conversations Among Information from
Different Sources (subscale C), indicating
that the treatment group papers were better
than the control group papers for this
subgroup. Out of the five subscales,
subscale C is essential, particularly in its
relationship to category E, which focuses on
students’ ability to enter the conversation in
a meaningful way. In order to enter the
conversation, students must first be able to
identify and articulate the conversations
already taking place, which is the focus of
Lundstrom et al, Teaching & Learning Info Synthesis Communications in Information Literacy 9(1), 2015
67
[ARTICLE]
Communications in Information Literacy, Vol. 9, Iss. 1 [2017], Art. 4
http://pdxscholar.library.pdx.edu/comminfolit/vol9/iss1/4
category C. Table 1 reports the analysis of
variance for all five subscales.
Synthesis Paragraph Analysis
At the end of the intervention, each group
had several information clusters on their
butcher paper (see figure 1). Every cluster
contained several sticky notes, each note
including a key point extracted from articles
read by the students. The sticky notes were
grouped together in topical clusters as part
of an iterative group process.
At the end of the lesson, student participants
were asked to pick a cluster and write a
paragraph synthesizing it. Students
submitted their synthesis paragraphs using a
web form. In the writing instructions for the
synthesized paragraph (provided in class
and also on the form), students were asked
to use the article ID (e.g., A, B, C) to cite an
article in the paragraph. The same web form
also collected additional data: student name,
group number, how many different articles
were contained on the sticky notes in the
cluster they used for writing their paragraph.
Students submitted 74 synthesized
paragraphs. After removal of paragraphs
without in-text citations, paragraphs based
on single-article clusters, and paragraphs
from students who missed the intervention,
a total of 52 paragraphs remained for the
analysis.
The 52 paragraphs were analyzed using
subscale C of the synthesis rubric, which
seemed to most robustly address the key
skill of making connections between
sources. No or very weak connections were
made for 9 (17.3%) paragraphs; only
implicit connections between the articles
were made for 10 (19.2%) paragraphs; some
explicit connections were made for 14
(26.9%); and several explicit connections
were made for many of the paragraphs (19
or 36.5%).
When comparing the level of synthesis in
Lundstrom et al, Teaching & Learning Info Synthesis Communications in Information Literacy 9(1), 2015
68
[ARTICLE]
Subscale F p-value
A - Source Variety 0.31 0.58
B - Using information from sources effectively 1.40 0.24
C - Identifies conversations among information from
different sources
7.36 0.01*
D - Organizes sources overall in a meaningful,
purposeful way
1.02 0.32
E - Analyzes sources to create something new or draw
conclusions and make generalizations
1.58 0.21
TABLE 1—ANALYSIS OF VARIANCE FOR THE RUBRIC’S FIVE SUBSCALES
*Significant at p<.05.
Lundstrom et al.: Teaching and Learning Information Synthesis: An Intervention and
Published by PDXScholar, 2017
the paragraphs with the level of synthesis
found in student papers, the paragraphs
fared much better in the highest score
category of subscale C (Identifies
conversations among information from
different sources). This remained true after
adding paragraphs without text citations
back into the analysis (see Table 2). This
finding suggests that our intervention does
help students with the process of
synthesizing at the paragraph level, but that
the process does not necessarily transfer to
the same extent to a larger scale project,
such as an entire paper.
Other methods for measuring
synthesis
After scoring each paper using the rubric,
the researchers wanted to know if there
were other identifiers of synthesis, such as
how many times a student used a source
across paragraphs. With this in mind, trends
were identified using a counting analysis
developed by the researchers. First, each
source was highlighted and numbered in the
works cited/bibliography list with a
different color. Second, each source used in
the paper was highlighted in the matching
color. A source could have been used in the
paper, but not listed in the reference list.
These sources were counted and added to
the works cited list, using the author or title,
if available. Third, a comment was added
next to each source noting if it was quoted
or referenced or both. Fourth, the number of
sources used in each paragraph was counted
(i.e., source #2 was used in three different
paragraphs). And fifth, the researchers
counted how many paragraphs from each
essay used one source, how many used two
sources, and so on (see Table 3).
General counting observations
As can be seen from Table 3, the control
and the treatment papers had the same
percentage of paragraphs with the same
number of sources. Over one-half (56%) of
the paragraphs citied no sources, while
Lundstrom et al, Teaching & Learning Info Synthesis Communications in Information Literacy 9(1), 2015
69
[ARTICLE]
FIGURE 1—INFORMATION FROM PAPERS ORGANIZED IN NAMED CLUSTERS
Communications in Information Literacy, Vol. 9, Iss. 1 [2017], Art. 4
http://pdxscholar.library.pdx.edu/comminfolit/vol9/iss1/4
almost one-third (29%) cited only one
source. However, having a large number of
single-source paragraphs does not
necessarily mean the paper scored low on
synthesis overall, as evidenced by control
group paper 046-12. This paper received
one of the two highest synthesis rubric totals
(score 10), but had nine paragraphs with
only a single source in them. In contrast, the
other high scorer on the synthesis rubric,
paper 046-17 (score 10), had three
paragraphs with two sources in them and
one paragraph with four sources, which
could indicate a high level of synthesis.
Source use
A common scenario for students was to
include a number of sources in the
references page that were not referenced or
quoted in the paper. In this case, use refers
to any time a source was referenced either
through paraphrasing or by direct quoting.
Lundstrom et al, Teaching & Learning Info Synthesis Communications in Information Literacy 9(1), 2015
70
[ARTICLE]
Writing Products N Rubric
Score C0
Rubric
Score
C1
Rubric
Score
C2
Rubric
Score
C3
Student Papers (no
intervention)
36 4 (11.1%) 22 (61.1%) 10 (27.8%) 0 (0%)
Student Papers
(intervention)
36 2 (5.6%) 12 (33.3%) 22 (61.1%) 0 (0%)
Paragraphs (intervention) 52 9 (17.3%) 10 (19.2%) 14 (26.9%) 19 (36.5%)
All Paragraphs*
(intervention)
71 28 (39.4%) 10 (14.1%) 14 (19.7%) 19 (26.8%)
TABLE 2—RUBRIC SUBSCALE C SCORES OF STUDENT WRITING PRODUCTS
*All paragraphs include paragraphs based on single article clusters (3) and paragraphs without any in-text
citations (16). All these 19 articles received a rubric score of 0.
% of
paragraphs
with 0
sources
% of
paragraphs
with 1
source
% of
paragraphs
with 2
sources
% of
paragraphs
with 3
sources
% of
paragraphs
with 4+
sources
Total #
Paragraphs
Control 56% 29% 11% 2% 2% 9.916
Treatment 56% 29% 11% 2% 2% 11.421
TABLE 3—NUMBER OF PARAGRAPHS WITH 0, 1, 2, 3, AND 4 OR MORE
SOURCES CITED
Lundstrom et al.: Teaching and Learning Information Synthesis: An Intervention and
Published by PDXScholar, 2017
Most students only used a single source per
paragraph; there were 119 instances of this
for the control group and 161 instances for
treatment group papers. The treatment group
used 54 more sources total than the control
group.
Papers that used one source often−an
instance of a source being used in three or
more different paragraphs−tended to have
lower synthesis scores (mode=5). From the
control group, paper 049-20 with a rubric
score of nine, and paper 046-12 with a score
of 10 were the exceptions. These students
tended to use one source frequently, but
they had higher synthesis scores. From the
students in the treatment group, most papers
using one source in three or more
paragraphs scored a six or seven from the
rubric.
In general, students paraphrased their
sources much more often than quoting. In
total, students from the control group quoted
their sources 60 (26%) times and
paraphrased them 167 (74%) times, while
students with the treatment group quoted 70
(23%) times and paraphrased 233 (77%)
times (see Table 4). In terms of information
synthesis, this indicates that students from
the treatment group are using multiple
sources in their papers. and they use the
information from those sources in a more
advanced way, paraphrasing instead of
directly quoting. These two areas are
covered by subscale A and B in the rubric.
Comparing the synthesis rubric and
the counting analysis
While the number of sources per paragraph
did not necessarily predict a high or low
score on the synthesis rubric as a whole,
there was a connection between papers that
scored a two or higher in subscale C from
the rubric; they all had at least one
paragraph with two sources.
There is a higher correlation between the
counting analysis and the subscale C
consensus score for the treatment papers
(r=0.74) versus the control papers (r=0.63).
Occasionally higher scores had more
sources used, however there were also
instances where there was no correlation
between high rubric scores and high number
of sources used. For example, one of the
papers only used three sources and scored
an 8 on the rubric. From the treatment
group, another paper used four sources and
scored an 8 on the synthesis rubric. This
indicates that number of sources used does
not necessarily correlate with a student’s
ability to synthesize that information.
Lundstrom et al, Teaching & Learning Info Synthesis Communications in Information Literacy 9(1), 2015
71
[ARTICLE]
Source
Quoted
Source
Referenced
used
in 1
used
in 2
used
in 3
used
in 4
used
in 5+
Total # of
Sources
Control Total 60 167 119 29 8 0 0 156
Treatment Total 70 233 161 34 11 3 1 210
TABLE 4—SOURCES QUOTED VS REFERENCED AND USED ACROSS
MULTIPLE PARAGRAPHS
Communications in Information Literacy, Vol. 9, Iss. 1 [2017], Art. 4
http://pdxscholar.library.pdx.edu/comminfolit/vol9/iss1/4
DISCUSSION
All three assessment techniques−the rubric
assessment of the paragraphs from the
intervention, the rubric assessment of the
final papers in comparison to the control
group, and the counting analysis of how
sources were used within final
papers−indicate students struggle with
synthesis and benefit from teaching methods
that break down the different skills involved
in synthesis. Each of these techniques
revealed key student behaviors that can help
librarians and educators modify their
instruction practices.
These findings confirm the difficulty of the
synthesis process for students, detailed in
much of the previous literature (Howard,
Serviss, & Rodrigue, 2010). The findings
also show that teaching interventions in this
process can help students improve their
ability to synthesize. In this case, the
intervention slowed down this process for
students, and it helped students identify
main ideas and collaboratively group those
ideas with others. Interestingly, the skill of
being able to identify conversations among
different texts appeared to transfer to
students’ final papers. Students may have
also benefited from having explicit
definitions of synthesis, such as sharing and
discussing the specific elements of the
comprehensive synthesis rubric used in this
study.
Defining Synthesis & Rubric Creation
As detailed in the literature, definitions of
what synthesis is and what it looks like vary
widely. One of the major components of this
research emerged with the realization that
solid markers for measuring synthesis are
difficult to find, largely due to its
complexity and scope. The creation of this
rubric helped to pinpoint what synthesis
involves and what it looks like in a paper.
The researchers determined that five
categories are most significant when
identifying the presence of information
synthesis: mainly source variety (subscale
A), using information sources effectively
(subscale B), identifying conversations
among sources (subscale C), organizing
sources meaningfully (subscale D), and
analyzing sources to create something new
(subscale E). These categories and their
descriptions at each level vary from other
existing rubrics, although they are adapted
from pieces of existing ones.
During the rubric creation and revision
phase, the researchers continuously reached
for measurable language whenever possible.
For example, source variety (subscale A)
clarifies numerically how varied the sources
must be to be categorized at each level,
focusing particularly on whether the student
uses sources that speak to a variety of
perspectives rather than using sources that
all trend towards the same view. Subscale
B, using information sources effectively,
attempts to distinguish haphazard use of
sources from summarizing main ideas.
Subscale C, identifying conversations,
underwent the most revision during this
process, including defining “explicit” as
sources that included textual indicators (i.e.,
using terms such as “likewise” or “in
contrast”), or the sources are placed side by
side in the same paragraph. Subscale D
attempts to show variation in the
effectiveness of the organization of sources.
Subscale E differentiates between works
that make assumed generalizations and
those that come to well-reasoned logical
conclusions as a result of how they use and
Lundstrom et al, Teaching & Learning Info Synthesis Communications in Information Literacy 9(1), 2015
72
[ARTICLE]
Lundstrom et al.: Teaching and Learning Information Synthesis: An Intervention and
Published by PDXScholar, 2017
respond to their sources.
These carefully articulated categories, or
elements of synthesis, combined with
measurable and specific descriptions at each
level, helped the researchers to feel
confident in how they were assessing
students’ ability to synthesize. The
categories also helped the researchers to
better understand where an instructor or
librarian might intervene in a student’s
process and how to more carefully scaffold
activities that support students in areas in
which they struggle. The results of the inter-
rater reliability analysis reinforced
confidence in the reliability of the rubric, as
well as in the usefulness of the intervention.
Impact of the intervention on the
synthesis paragraphs
Judging by the synthesized paragraphs that
students wrote directly after the intervention
(scored on subscale C of the rubric), it
appears that the synthesis lesson helped
students with the process of synthesizing
information from difference sources into
paragraphs. This effect is much less clear
when looking at the final student papers.
One of the reasons for this could be that the
intervention combined individual thought
processes with group discussions, while the
final papers are individual efforts. Another
reason could be that the paragraphs were
written immediately after the intervention,
while the papers were not completed for
another two weeks. Additionally, several
students had already started on their final
papers before our intervention; they may
have been too far along in the writing
process to start over and fully apply the
synthesis process they learned in class.
Another aspect to consider is suggested by
Cavanaugh and Perlmutter (1982) who
noted that students tend to use strategies
when instructed to do so, but stop using
these strategies when they are no longer
required. Students in this study went
through the synthesis strategy as part of the
intervention and just before they wrote their
synthesis paragraphs. The instructions for
the final paper did not mention that students
should use the synthesis strategy. There are
two possible solutions for this in future
work. One would be to slowly fade the
scaffold, which would lengthen the
intervention. The other would be to create a
synthesis tool that would serve as a
cognitive support with the synthesis steps
built in across the entire research and
writing process. Pea (2004) calls this latter
approach a case of distributed intelligence,
where the intelligence is incorporated into
the software. Changing the intervention to
allow fading the scaffold should be possible
by using an informed learning approach
(Maybee, Bruce, Lupton, Rebmann, 2013;
Bruce and Hughes, 2010), where students
actively use information to learn instead of
being taught an information skill like
synthesis in isolation. The researchers are
currently experimenting with this approach
in an online class where students curate
content to create learning modules.
The impact of the intervention on
final papers
The majority of the control group papers
(from the Fall class that did not receive the
intervention) received a score of one on
subscale C, while the majority of the
treatment group papers submitted after the
intervention received a score of two. This
indicates that our intervention had an effect.
Interestingly, the synthesis scores for
submitted paragraphs are more evenly
Lundstrom et al, Teaching & Learning Info Synthesis Communications in Information Literacy 9(1), 2015
73
[ARTICLE]
Communications in Information Literacy, Vol. 9, Iss. 1 [2017], Art. 4
http://pdxscholar.library.pdx.edu/comminfolit/vol9/iss1/4
distributed across all four categories in each
subscale compared to the final paper scores,
which are mostly clustered in the two
middle scores. Conversely, papers that used
three or four sources in a paragraph did not
necessarily correlate to having higher rubric
scores.
Effectiveness of Counting Analysis
The counting method did reveal some
interesting patterns about how students use
sources. There does not appear to be a
correlation between the number of
paragraphs containing certain numbers of
cited sources and the total synthesis rubric
score. This might be because the individual
subscales are measuring something different
that cannot be captured by counting.
Also, when looking at how many times
sources were used across the submitted
paragraphs, there was a significant
difference between the control and the
experimental group on subscale C. The
rubric scores from the other categories did
not appear to relate significantly to the
counting analysis results. Subscale C did
correlate well with the rubric scoring
because the language of the rubric clarified
that papers that received a two in subscale C
had to have at least two sources in one
paragraph in conversation with one another.
While this analysis was fairly limited in
correlating numbers of sources to
identifying synthesis, more research is
needed to explore alternative ways of
measuring this skill. In general, this analysis
did help identify student behaviors relating
to synthesis. There were also quite a few
instances of sources listed in a reference
page that were not actually used in an
identifiable way in the paper. This may
reflect a tendency for student to place more
emphasis on finding and documenting
sources, rather than learning to use them
effectively to support their argument and
contribute something new to a conversation.
These findings indicate a continued need for
teaching synthesis, but also for revising
scaffolds, such as the intervention used in
this study, to target some of these specific
areas.
CONCLUSION
Despite the difficulty of learning and
teaching synthesis, breaking down the
processes involved in synthesis, both in
definition and in scaffolded practice, can
help instructors teach and assess synthesis
more effectively. In this study, the authors
created a comprehensive, reliable rubric that
clearly defined multiple aspects of synthesis
and helped to determine which elements of
synthesis students struggle with, as well as
identifying when synthesis was employed in
student work. Using this rubric, the
researchers determined that the attempts to
break down the synthesis process in the
intervention may have improved students’
ability to identify conversations within the
literature and incorporate those
conversations, in identifiable ways, into
their final persuasive research papers.
These findings are relevant for instructors
and for librarians as they support the
research process. The issue of whether
librarians should be involved in teaching
synthesis is an important one. As librarian
roles continue to shift, and as they
collaborate with faculty to find ways to help
students with the threshold concepts in the
Framework for Information Literacy for
Higher Education, librarians may be
Lundstrom et al, Teaching & Learning Info Synthesis Communications in Information Literacy 9(1), 2015
74
[ARTICLE]
Lundstrom et al.: Teaching and Learning Information Synthesis: An Intervention and
Published by PDXScholar, 2017
increasingly more involved in teaching
higher level concepts like synthesis. In this
study, a librarian taught the intervention, but
the lesson plan can also be shared by
librarians with instructors looking for ways
to help students learn synthesis. It is clear
from previous studies on student research
skills, such as reports from Project
Information Literacy (Head & Eisenberg,
2009), that appropriate usage of information
is a major area of difficulty for students.
Students and instructors may not recognize
that librarians can assist the synthesis
process, but as this study shows, they can.
Librarians must be clearer about their role in
helping with any facet of the research
process, including synthesis.
Future avenues for research include
adapting and creating more interventions
that breakdown the synthesis process,
particularly in relation to the five categories
identified in the rubric used for the present
study. In this case, the intervention targeted
identifying conversations, but more research
is needed in how to help students with the
other rubric elements. Further inquiry into
using the informed learning approach may
also help identify effective ways to scaffold
elements of synthesis. Alternative ways to
assess synthesis, such as the counting
analysis method attempted here, also
warrants more exploration. Methods like
these can help librarians and instructors
identify when synthesis is occurring and
what it looks like in order to teach students
effective strategies for synthesizing
information.
REFERENCES
Afflerbach, P., & Cho, B. (2009).
Identifying and describing constructively
responsive comprehension strategies in new
and traditional forms of reading. In S. Israel
& G. Duffy (Eds.), Handbook of research
on reading comprehension (pp. 69–90).
New York, NY: Routledge.
Andretta, S. (2009). Transliteracy: take a
walk on the wild side. Paper presented at the
World Library and Information Congress:
75th IFLA General Conference and
Assembly, Milan, Italy. Retrieved from
http://eprints.rclis.org/14868/
Asher, A., Duke, L., & Green, D. (2010).
The ERIAL project: Ethnographic research
in Illinois academic libraries. Academic
Commons, (May). Retrieved from http://
www.academiccommons.org/2014/09/09/
the-erial-project-ethnographic-research-in-
illinois-academic-libraries/
Association of American Colleges and
Universities (2010). Inquiry and Analysis
VALUE Rubric. Retrieved from https://
www.aacu.org/value/rubrics/inquiry-
analysis
Association of College & Research
Libraries (2014). Framework for
information literacy for higher
education (draft). Retrieved from http://
acrl.ala.org/ilstandards/wp-content/
uploads/2014/11/Framework-for-IL-for-HE-
draft-3.pdf
Blake, C., & Pratt, W. (2002). Collaborative
information synthesis. Paper presented at the
Meeting of the American Society of
Information Science and Technology
(ASIS&T), Philadelphia, PA.
Bloom, B. S. (1956). Taxonomy of
educational objectives: the classification of
Lundstrom et al, Teaching & Learning Info Synthesis Communications in Information Literacy 9(1), 2015
75
[ARTICLE]
Communications in Information Literacy, Vol. 9, Iss. 1 [2017], Art. 4
http://pdxscholar.library.pdx.edu/comminfolit/vol9/iss1/4
educational goals. New York, NY:
Longmans, Green.
Bråten, I., & Strømsø, H. I. (2003). A
longitudinal think-aloud study of
spontaneous strategic processing during the
reading of multiple expository texts.
Reading and Writing: An Interdisciplinary
Journal, 16(3), 195-218. doi: 10.1023/
A:1022895207490
Britt, M. A., & Aglinskas, C. (2002).
Improving students’ ability to identify and
use source information. Cognition and
Instruction, 20(4), 485–522.
Bruce, C., & Hughes, H. (2010). Informed
learning: A pedagogical construct attending
simultaneously to information use and
learning. Library & Information Science
Research, 32(4), A2-A8. doi:10.1016/
j.lisr.2010.07.013
Cavanaugh, J. C., & Perlmutter, M. (1982).
Metamemory: A critical examination. Child
Development, 53, 11-28. doi:
10.2307/1129635
Eisenberg, M., & Berkowitz, R. E. (1990).
Information problem-solving: The Big Six
Skills approach to library & information
skills instruction. Norwood, N.J.: Ablex
Pub. Corp.
Fitzgerald, M. A. (2004). Making the leap
from high school to college: Three new
studies about information literacy skills of
first-year college students. Knowledge
Quest, 32(4). Retrieved from http://
www.libs.uga.edu/cloc/readings/
fitzgerald.pdf
Flower, L., Stein, V., Ackerman, J., Kantz,
M. J., McCormick, K., & Peck, W. C.
(1990). Reading-to-write: Exploring a
cognitive and social process. New York:
Oxford University Press.
Fluellen, J. E., Jr. (2011). Connecting
minds: Emergence of a future bent, writing/
thinking intensive psychology course.
Occasional Paper #10. Retrieved from
http://files.eric.ed.gov/fulltext/
ED521161.pdf
Ford, T. Synthesis Paper Rubric. Resources
for first year and learning communities.
Retrieved from http://
academic.evergreen.edu/f/fordter/
synthrubric.htm
Goldman, S. R. (2004). Cognitive aspects of
constructing meaning through and across
multiple texts. In N. Shuart-Faris & D.
Bloome (Eds.), Uses of intertextuality in
classroom and educational research (pp.
317–351). Greenwich: Information Age
Pub.
Goldman, S. R., Braasch, J. L. G., Wiley, J.,
Graesser, A. C., & Brodowinska, K. (2012).
Comprehending and learning from Internet
sources: Processing patterns of better and
poorer learners. Reading Research
Quarterly, 47(4), 356-381. doi: 10.1002/
rrq.027
Goldman, S. R., & Scardamalia, M. (2013).
Managing, understanding, applying, and
creating knowledge in the information age:
Next-generation challenges and
opportunities. Cognition and Instruction, 31
(2), 255-269. doi: 10.1080/
10824669.2013.773217
Goldschmidt, P. G. (1986). Information
Lundstrom et al, Teaching & Learning Info Synthesis Communications in Information Literacy 9(1), 2015
76
[ARTICLE]
Lundstrom et al.: Teaching and Learning Information Synthesis: An Intervention and
Published by PDXScholar, 2017
synthesis: A practical guide. Health Services
Research, 21(2 PT 1), 215-237. Retrieved
from http://www.ncbi.nlm.nih.gov/pmc/
articles/PMC1068946/
Head, A. J., & Eisenberg, M. B. (2009).
Lessons learned: How college students seek
information in the digital age. Project
Information Literacy (PIL) at the University
of Washington's Information School,
Seattle, WA. Retrieved from http://
projectinfolit.org/images/pdfs/
pil_fall2009_finalv_yr1_12_2009v2.pdf
Howard, R. M., Serviss, T., & Rodrigue, T.
K. (2010). Writing from sources, writing
from sentences. Writing & Pedagogy, 2(2),
177-192. doi: 10.1558/wap.v2i2.177
Johnson, C. R. (2003). Synthesizing
information. Workshop materials. Arizona
State University West. Glendale, AZ.
Retrieved January 24, 2012 from http://
www.west.asu.edu/johnso/synthesis/
synthesis.html#examples
Johnson, J. D. (2009). An impressionistic
mapping of information behavior with
special attention to contexts, rationality, and
ignorance. Information Processing &
Management, 45(5), 593-604. doi: 10.1016/
j.ipm.2009.04.005
Jucks, R., & Paus, E. (2013). Different
words for the same concept: Learning
collaboratively from multiple documents.
Cognition and Instruction, 31(2), 227-254.
doi: 10.1080/07370008.2013.769993
Klassen, P.T. (2014). General Education
Assessment Rubrics. Retrieved from http://
documentingexcellence.com/examples/
rubricgened/rubric.htm
Kolowich, S. (2010, October 18). Searching
for better research habits. Inside Higher Ed.
Retrieved from https://
www.insidehighered.com/news/2010/09/29/
search
Krathwohl, D. R. (2002). A revision of
Bloom's taxonomy: An overview. Theory
into Practice, 41(4), 212-218. doi: 10.1207/
s15430421tip4104_2
Kuhlthau, C. C. (1991). Inside the search
process: information seeking from the user's
perspective. Journal of the American
Society for Information Science, 42(5), 362-
371.
Kuhlthau, C. C., Heinström, J. & Todd, R.J.
(2008). The 'information search process'
revisited: is the model still useful?
Information Research, 13 (4).
Larsen, R., Wactlar, H., Friedlander, A., &
University of Pittsburgh. (2003). Knowledge
lost in information: Report of the NSF
workshop on research directions for digital
libraries, June 15-17, 2003, Chatham, MA.
Pittsburg, PA: University of Pittsburg,
School of Information Sciences in
cooperation with the Department of
University Marketing Communications.
Lloyd, A. (2007). Recasting information
literacy as sociocultural practice:
Implications for library and information
science researchers. Information Research,
12, 1-13.
Lowe, C. A., & Eisenberg, M. B. (2005).
Big 6TM Skills for information literacy. In K.
E. Fisher, S. Erdelez & L. McKechnie
(Eds.), Theories of information behavior
(pp. 63-68). Medford, NJ: Information
Lundstrom et al, Teaching & Learning Info Synthesis Communications in Information Literacy 9(1), 2015
77
[ARTICLE]
Communications in Information Literacy, Vol. 9, Iss. 1 [2017], Art. 4
http://pdxscholar.library.pdx.edu/comminfolit/vol9/iss1/4
Today.
Macedo-Rouet, M., Braasch, J. L. G., Britt,
M. A., & Rouet, J.-F. (2013). Teaching
fourth and fifth graders to evaluate
information sources during text
comprehension. Cognition and Instruction,
31(2), 204-226. doi:
10.1080/07370008.2013.769995
Mateos, M., Martin, E., Villalon, R., &
Luna, M. (2008). Reading and writing to
learn in secondary education: Online
processing activity and written products in
summarizing and synthesizing tasks.
Reading and Writing: An Interdisciplinary
Journal, 21(7), 675–697. doi: 10.1007/
s11145-007-9086-6
Mateos, M., & Sole, I. (2009). Synthesising
information from various texts: A study of
Procedures and products at different
educational levels. European Journal of
Psychology of Education, 24(4), 435-451.
doi: 10.1007/BF03178760
Maybee, C., Bruce, C. S., Lupton, M., &
Rebmann, K. (2013). Learning to use
information: Informed learning in the
undergraduate classroom. Library &
Information Science Research, 35(3), 200-
206. doi: 10.1016/j.lisr.2013.04.002
McGinley, W. (1992). The role of reading
and writing while composing from multiple
sources. Reading Research Quarterly, 27
(3), 227-248.
McGregor, J. (2011). A Visual Approach:
Teaching Synthesis. School Library
Monthly, 27(8), 5-7. Retrieved from http://
www.schoollibrarymonthly.com/articles/
Mcgregor2011-v27n8p5.html
Montiel-Overall, P. (2007). Information
literacy: Toward a cultural model. Canadian
Journal of Information and Library Science,
31(1), 43-68.
National Science Board. (2005). Long-lived
digital data collections : enabling research
and education in the 21st century.
Arlington, VA: National Science
Foundation. Retrieved from http://
www.nsf.gov/pubs/2005/nsb0540/
Nelson, J., & Hayes, J. R. (1988). How the
writing context shapes college students’
strategies for writing from sources.
Berkeley, CA: Center for the Study of
Writing, Carnegie Mellon University.
Northern Arizona University. Synthesis
Essay Rubric. Retrieved from http://
www2.nau.edu/d-elearn/assessment/
documents/Essay_2_rubric.pdf
Pea, R. D. (2004). The social and
technological dimensions of scaffolding and
related theoretical concepts for learning,
education, and human activity. Journal of
the Learning Sciences, 13(3), 423-451. doi:
10.1207/s15327809jls1303_6
Perfetti, C. A., Rouet, J.-F., & Britt, M. A.
(1999). Toward a theory of documents
representation. In H. v. Oostendorp & S. R.
Goldman (Eds.), The construction of mental
representations during reading (pp. 99–
122). Mahwah, NJ: Lawrence Erlbaum
Associates.
Oakleaf, Megan J. (2006). Assessing
information literacy skills: A rubric
approach. Retrieved from http://
www.meganoakleaf.info/
oakleafdissfrontpapers.pdf
Lundstrom et al, Teaching & Learning Info Synthesis Communications in Information Literacy 9(1), 2015
78
[ARTICLE]
Lundstrom et al.: Teaching and Learning Information Synthesis: An Intervention and
Published by PDXScholar, 2017
Rouet, J.-F. (2006). The skills of document
use: from text comprehension to Web-based
learning. Mahwah, NJ: L. Erlbaum
Associates.
Rubric Assessment of Information Literacy
Skills (RAILS): Using Information to
Accomplish a Purpose (2011). Assessment
Immersion. Retrieved from http://
railsontrack.info/rubrics_details.aspx?
wkid=181&catid=12
Simmons, M. H. (2005). Librarians as
disciplinary discourse mediators: Using
genre theory to move toward critical
information literacy. portal: Libraries and
the Academy, 5(3), 297-311. doi: 10.1353/
pla.2005.0041
Spivey, N.N. (1984). Discourse synthesis:
Constructing texts in reading and writing.
(Outstanding Dissertation Monograph
Series). Newark, DE: International Reading
Association.
Spivey, N. N., & King, J. R. (1989).
Readers as writers composing from sources.
Reading Research Quarterly, 7-26.
Stadtler, M., & Bromme, R. (2013).
Multiple document comprehension: An
approach to public understanding of science.
Cognition and Instruction, 31(2), 122-129.
doi: 10.1080/07370008.2013.771106
Stadtler, M., Scharrer, L.,
Brummernhenrich, B., & Bromme, R.
(2013). Dealing with uncertainty: Readers'
memory for and use of conflicting
information from science texts as function
of presentation format and source expertise.
Cognition and Instruction, 31(2), 130-150.
doi: 10.1080/07370008.2013.769996
Stripling, B. (2010). Teaching students to
Think in the digital environment: Digital
literacy and digital inquiry. School Library
Monthly, 26(8), 16-19. Retrieved from
http://www.schoollibrarymonthly.com/
articles/Stripling2010-v26n8p16.html
Strømsø, H. I., Bråten, I., Britt, M. A., &
Ferguson, L. E. (2013). Spontaneous
sourcing among students reading multiple
documents. Cognition and Instruction, 31
(2), 176-203. doi:
10.1080/07370008.2013.769994
Tavakol, M., & Dennick, R. (2011, June).
Making sense of Cronbach’s alpha.
International Journal of Medical Education,
2, 53-55. doi: 10.5116/ijme.4dfb.8dfd
Thomas, S., Joseph, C., Laccetti, J., Mason,
B., Mills, S., Perril, S., & Pullinger, K.
(2007). Transliteracy: Crossing divides.
First Monday, 12(12). doi: 10.5210/
fm.v12i12.2060
Tooley, M. (2005). Big6 TurboTools and
synthesis. Library Media Connection, 24(2),
30-31.
Torraco, R. J. (2005). Writing Integrative
Literature Reviews: Guidelines and
Examples. Human Resource Development
Review, 4(3), 356-367.
Tuominen, K., Savolainen, R., & Talja, S.
(2005). Information Literacy as a
sociotechnical practice. Library Quarterly,
75(3), 329-345.
Webber, S. A. E., & Johnston, B. (2000).
Conceptions of information literacy: new
perspectives and implications. Journal of
Information Science, 26(6), 381-397.
Lundstrom et al, Teaching & Learning Info Synthesis Communications in Information Literacy 9(1), 2015
79
[ARTICLE]
Communications in Information Literacy, Vol. 9, Iss. 1 [2017], Art. 4
http://pdxscholar.library.pdx.edu/comminfolit/vol9/iss1/4
Wineburg, S. S. (1991). Historical problem
solving: A study of the cognitive processes
used in the evaluation of documentary and
pictorial evidence. Journal of Educational
Psychology, 83(1), 73-87. doi:
10.1037/0022-0663.83.1.73
Wolfe, M. B. W., & Goldman, S. R. (2005).
Relations between adolescents' text
processing and reasoning. Cognition and
Instruction, 23(4), 467-502. doi: 10.1207/
s1532690xci2304_2
Lundstrom et al, Teaching & Learning Info Synthesis Communications in Information Literacy 9(1), 2015
80
[ARTICLE]
Lundstrom et al.: Teaching and Learning Information Synthesis: An Intervention and
Published by PDXScholar, 2017
Lundstrom et al, Teaching & Learning Info Synthesis Communications in Information Literacy 9(1), 2015
81
APPENDIX A—SYNTHESIS RUBRIC
Definition from
lesson
Not present or
unacceptable = 0
Needs Improvement
= 1
Developing = 2
Advanced/Mastery = 3
A. Source Variety
Does not use
multiple sources
(<2). Sources do
not need to be in a
reference list; any
mention of any
outside source
works.
Uses a few different
sources, but with little
variation (1 source of
variation) in
perspective, trending
toward the same
view. Sources do not
need to be in a
reference list; any
mention of any
outside source works.
Uses a variety of
sources that cover some
of the differing
perspectives (2 or more
perspectives). Sources
do not need to be in a
reference list; any
mention of any outside
source works.
Uses multiple sources which
address multiple
perspectives, including
opposing viewpoints.
Sources do not need to be in
a reference list; any mention
of any outside source works.
B. Using
information from
sources
effectively
Does not present
information from
sources. No in-text
citations present.
Uses information
from sources with no
added value, with
little or no summary -
used only as support
(haphazard, provide
too much or too little
information, and/or
serve no clear
purpose).
Uses information from
sources through direct
quotes and/or
paraphrasing and begins
to summarize main
ideas (making main
ideas more clear and
succinct; implicit
connections to the thesis
or main ideas of the
paper).
Uses information from
sources through direct quotes
when necessary,
paraphrasing, summarizing,
and explicit connections
(making the main ideas clear,
succinct, and connected to
the thesis or main ideas in
the paper).
C. Identifies
conversations
among
information from
different sources
No or very weak
connections are
made between
sources. The
reader cannot see
how the sources
are related to each
other.
Few implicit
connections are made
between sources.
Similarities,
differences,
relationships and
patterns are rarely
identified so it is
difficult for the reader
to see how the
sources are related.
There are some explicit
connections between
sources (textual
indicators or side by
side in paragraph
connections).
Similarities, differences,
relationships, and
patterns are sometimes,
but not consistently,
identified so the reader
can see how some of the
sources are related.
There are several explicit
connections between sources
including connections
between contradictory
sources. Similarities,
differences, relationships,
and patterns are almost
always identified so the
reader can see how the
sources are related and how
they support the thesis.
D. Organizes
sources overall in
a meaningful,
purposeful way
Does not use any
information from
sources and lacks
organization.
Uses information
from sources and
attempts to organize
information but the
organization is not
effective in revealing
important patterns,
differences or
similarities. (AACU)
Uses information from
sources and effectively
organizes information to
reveal some important
patterns, differences or
similarities to focus.
(AACU)
Uses information from
sources and effectively
organizes information to
reveal insightful patterns,
differences, or similarities
related to focus. (AACU)
E. Analyzes
sources to create
something new or
draw conclusions
and make
generalizations
No conclusions or
generalizations
offered
(Evergreen).
Author does not
attempt to make
sense of the
information he/she
uses.
The relationship
between the ideas in
sources and the
author's conclusions
or implicit
generalizations are
assumed or unclear.
The author uses
information sources to
come to some
reasonable
generalizations or well-
reasoned conclusions.
(Ford, T.)
The author relates
knowledge from several
areas/sources of information
in order to demonstrate
comprehension, make
insightful analyses, and draw
clear conclusions.
Communications in Information Literacy, Vol. 9, Iss. 1 [2017], Art. 4
http://pdxscholar.library.pdx.edu/comminfolit/vol9/iss1/4
Lundstrom et al, Teaching & Learning Info Synthesis Communications in Information Literacy 9(1), 2015
82
[ARTICLE]
APPENDIX B—DETAILED LESSON PLAN: TEACHING INFORMATION
SYNTHESIS (75 MINUTES)
Learning Outcomes
Students will be able to:
1. Analyze texts in order to identify important facts, concepts, and questions for further research.
2. Consider the ideas from multiple sources of information in order to infer relationships between concepts, facts, and
arguments.
3. Organize ideas from multiple sources in order to create meaningful groups of related concepts.
4. Restate facts and concepts in their own words in order to communicate a new understanding of the relationships
between ideas from multiple sources.
I. Introduction (5 minutes)
1. What we’re doing today and why (opportunity to opt out of being part of the study)
2. Final product: graded, posted synthesized paragraph
3. Synthesis Defined
II. Guided Practice:
Context for Today (1 minute):
“One of your family members has been diagnosed with depression. You know very little about depression, the possible
causes or treatment options. You decide to do a little background reading to familiarize yourself with this topic so
you can provide better support.”
2. Have students form teams of 3 to read articles. (8 minutes)
a. Individually they will read 2 short articles – 1 shared, 1 distinct
b. Students will highlight the following things:
i. What seems important or meaningful
ii. Points they could explain, share
iii. Points that need clarification
3. Have students share what they learned as a Group (10 minutes)
a. Share what they learned
b. Ask questions for understanding
c. Teach each other
4. Organizing Information Individually (8 minutes)
a. Individual clustering using post-its
i. Pull out key points – highlighted parts
ii. Put article letter in corner of post-it (A or B)
iii. Five post-its/main ideas per article
iv. One idea per post-it (about 3-5 words); use large letters
5. Organizing Information as a Group (10 minutes)
a. As a team, group like post-its together
i. Start with a few, add a few more
ii. Work until all post-its are in a cluster
b. As a team, name the clusters
i. 1 or 2 word nickname
ii. Must fit EVERY post-it in the cluster
6. Reviewing your information as a Group (5 minutes)
a. Narrowing (using big clusters as main themes, splitting clusters)
b. Broadening (drop clusters out as too minor, return to articles to get more info., return to
gathering phase to find more articles)
III. Wrap-up & Making Connections as a Group (8 minutes)
a. Creating an Outline
b. Avoiding Plagiarism
IV. In-Class Assignment Individually (20 minutes)
a. Put articles aside.
b. Students will post in Canvas before class ends one paragraph using one cluster of main ideas
from their organized post-its. Students can use in-text citations for articles A, B, etc. where needed
Lundstrom et al.: Teaching and Learning Information Synthesis: An Intervention and
Published by PDXScholar, 2017
