Grant Summary

Guru: a computer tutor that models expert human tutors
Institute of Education Sciences Award Number R305A080594

IES-NCER-2008-01 Education Technology Goal Two (Development)
The purpose of the proposed research is to develop Guru, an expert computer tutor, by modeling
the strategies and dialogue of expert human tutors. Expert human tutors promote larger learning
gains than novice human tutors. The proposing team has previously built and evaluated a novice
computer tutor that performs as well as novice human tutors. The Guru expert tutor, by using
expert human tutor strategies, actions, and dialogue, should promote larger learning gains than
previous novice computer tutors.

In future Goal 3 efficacy studies, Guru could be used to further

our understanding of the processes and mechanisms of expert tutoring by manipulating strategies
and dialogue moves and observing student learning outcomes. The setting is urban school
districts in Memphis, Tennessee. The population will consist of 9th grade students in Memphis
City Schools (MCS). 87% of students enrolled in MCS are African-American, 9% of students are
white, and 71% of students are eligible for a free or reduced lunch. 99% of MCS schools are
Title 1, 79% are economically disadvantaged, and the MCS graduation rate is 67%. MCS science
proficiency in K-8 science is an F and 75% below proficiency on the 8th grade, 2005 NAEP.

The
developed intervention will consist of an expert computer tutor for biology. Specifically, the
Guru expert tutor will improve educational outcomes on the Tennessee Gateway Science Test,
which students must pass in order to receive a high school diploma. The primary research
method is a design research, using the Integrative Learning Design Framework (ILDF), and
randomized controlled trials to evaluate the artificial intelligence components of the system. A
design experiment methodology is appropriate because of its focus on the design,
implementation, and systematic study, in real world settings, of an artifact embodying an
educational theory. Outcome measures for the proposed work exist on different levels of
aggregation. Individual software components will be measured using logistic regression and
likelihood ratios according to how well they predict features of the expert tutoring data. Cycles
of think-aloud protocols and eye tracking data from the target population will be collected and
used to evaluate the usability and feasibility of the system, as well as to gather more information
about users and needs. Expert biology tutors will also aid in evaluation and will provide feedback
on the biology content as well as on usability and feasibility issues. A final summative evaluation
will be conducted with the refined system. The summative evaluation will involve a small,
representative sample of MCS students who will participate in a randomized controlled
experiment. The summative evaluation will provide preliminary evidence of Guru’s
effectiveness on student learning outcomes and will demonstrate the readiness of the system for
future efficacy studies. The data analytic strategy will investigate the manner in which expert
human tutors select dialogue moves and dynamically plan pedagogical strategies in response to
student behavior and inferred mental state. Annotated data will be used to create artificial
intelligence models using machine learning techniques, including dimensionality reduction
methods and sequence extraction methods. Data from interaction studies with expert human
tutors, educators, and students will be used to verify that dialogues with the Guru expert tutor are
consistent with that of a human expert tutor. This will be accomplished using correlational
analyses of dialogue as well as bystander Turing tests in which a human makes judgments on
whether dialogue was generated by a human or computer.

Comments