EXAMPLE REVIEW JOURNAL
A.
Research Question
1.
Is the Reasoning Mind program more
engaging than traditional classroom instruction?
2.
Is the Genie 3 platform more
engaging than Genie 2?
B.
Basic Theory
Particularly
in real-world settings where educational software must compete with many other
activities, determining whether or not design features are engaging students is
a necessary component of evaluating their effectiveness, leading many to
investigate behavioral and affective indicators of student engagement in STEM
learning systems.
While
findings suggest that some time off-task can refocus bored or frustrated
students (Baker et al. 2011; Sabourin et al. 2011), students who completely
disengage with educational software, spending large amounts of time off task,
show lower learning (Goodman 1990), both in the shortterm and in the long-term,
the latter a result of aggregate effects from a loss of practice opportunities
(Cocea et al. 2009).
Other
disengaged behaviors such as carelessness and gaming the system are also
associated with poor learning outcomes (Cocea et al. 2009; Pardos et al.2014).
Furthermore, findings suggest engaged concentration (or
Csikszentmihalyi’s“flow”)is positively associated with learning, while boredom
leads to poor learning outcomes (Craig et al. 2004; D’Mello and Graesser 2012;
Pardos et al. 2014).
Confusion
and frustration have more complex relationships to learning; while necessary
for learning (D’Mello et al. 2014), spending a considerable amount of time
confused or frustrated is associated with worse outcomes (e.g., Liu et al.
2013).
One
of the more comprehensive discussions of designing for multimedia learning
systems has been put forward by Clark and Mayer (2011), who present eight
principles based on previous research.
These include the following:
(1)
Personalization: Use a conversational style, polite speech, and virtual coaches
(Moreno et al. 2001).
(2)
Multimedia: Use words and graphics, not words alone (Halpern et al. 2007).
(3)
Contiguity: Align words to corresponding graphics (Moreno and Mayer 1999).
(4)
Coherence: Limit extraneous information (Mayer et al. 2001).
(5)
Modality: Present words as audio, rather than text (Low and Sweller 2005).
(6)
Redundancy: Explain visuals with spoken word or text, not both (Mayer and
Moreno 2003).
(7)
Segmenting:Present lessons in small, well-spaced units (Mayer and Chandler
2001).
(8)
Pretraining: Ensure that learners know the names and characteristics of key
concepts (Kesteret al. 2006).
Each
of these principles is designed to enhance learning by focusing
students’attention and limiting cognitive load.
Using
design to focus the students’attention on the critical task of learning
mathematics should minimize the attentional resources needed to inhibit
distractions (Mayer and Moreno 2003), allowing for greater and more prolonged
attention to be paid to learning. In this paper, we investigate whether design
changes that reflect three of Clark and Mayer’s(2011)
principles—Personalization, Modality,and Redundancy—improve student engagement
with an online STEM learning system, Reasoning Mind.
Developed
by the nonprofit company of the same name, Reasoning Mind currently provides
blended learning instruction in mathematics to over 100,000 students in the
United States. Reasoning Mind works with
expert teachers to design online learning experiences that recreate
best-practices for instruction (Khachatryan et al. 2014), providing elementary
and middle school curricula that focuses on fostering deep understanding of
core mathematical topics necessary for students’ later success in algebra.
Engagement is a concept that has
been defined in many ways (see review in Fredricks et al. 2004). Finn and
Zimmer (2012) outline four components of engagement thought to impact student
learning and achievement: academic, social, cognitive, and affective.Both
academic and social engagements are comprised of behavioral indicators (treated
as a single construct in Fredricks et al. 2004).
The former refers to behaviors
related to the learning process, while the latter reflects whether or not the
student follows written and unwritten rules for classroom behavior. Cognitive
engagement involves the use of mental resources to comprehend complex ideas.Affective
engagement is the emotional response and feelings of involvement in school.Previous
research has often examined these constructs using survey methods.
For example, Finn et al. (1991) administered
a questionnaire to teachers, finding that academic behaviors that reflect
effort and initiative are positively correlated with end of year achievement
test scores (r=0.40 to 0.59), while inattentive behavior is negatively
correlated with achievement (r=−0.52 to−0.34). More recently, research has
found a significant relationship between academic and social engagement in
fourth and eighth grades and high-school graduation (Finn and Zimmer 2012).
C.
Method
In this paper, student engagement
measures (discussedmore thoroughly in the next section) are investigated in
series of three field observation studies that investigate the effect of the
Reasoning Mind mathematics curricula on the prevalence of these indicators.
Study 1 reports on observations of
student engagement that were conducted when students were using the Genie 2
platform.
Study 2 uses the same observation
method to compare the engagement of students using Genie 2 to those using Genie
3.
Finally,
study 3 compares students using Genie 3 to students in a traditional
mathematics classroom (with no technological support).
D. Results:
Study 1 found very high level sof student engagement with the
Genie 2 platform, with 89% time on task and 71% engaged concentration. Study 2
found that students using Genie 3 spent sign ificantly more time in independent
on-task behavior and less time off- task or engaged in on task conversation
with peers than students using Genie 2. Students using Genie 3 also showed more
engaged concentration and less confusion. Study 3 found that students using
Genie 3 spent 93% of their time on-task,compared to 69% in traditional
classrooms. They also showed more engaged concentration and less boredom and
confusion. Genie 3 students sustained their engagement for the entire class
period,while engagement in the traditional class room dropped off later in the
class session. In both study 2 and 3,Genie 3 students showed more growth from
pre-to post-test on an assessment of key concepts in sixth-grade mathematics.
E. Conclusions:
The in corporation of evidence – based –learning principles in
to the design of the Genie 3 platform resulted in higher level sof student
engagement when compared to an earlier,well-established platform that lacked
those principles,as well as when compared to traditional classroom
instruction.Increased personalization ,the use of multiple modalities,and
minimization of redundancy resulted in significant increases in time on-task
and engaged concentration ,but also a decrease in peer interaction.On the
whole,this evidence suggests that capturing students’attention,fostering deep
learning,and minimizing cognitive load lead stoim proved engagement,and ultimately
better education al out comes.
F.
References
Mulqueeny, Kevin et.all,2015, Incorporating effective e-learning
principles to improve student engagement in middle-school mathematics.International
Journal of STEM Education. DOI 10.1186/s40594-015-0028-6. https://stemeducationjournal.springeropen.com. 9 September 2016
Comments
Post a Comment