Electronic Theses and Dissertations

Identifier

2464

Date

2015

Document Type

Dissertation

Degree Name

Doctor of Philosophy

Major

Computer Science

Committee Chair

Vasile Rus

Committee Member

Arthur C. Graesser

Committee Member

Lan Wang

Committee Member

Scott D. Fleming

Abstract

Dialogue-based Intelligent Tutoring Systems (ITSs) have already proven to be very effective at inducing learning gains in students. These systems are guided by dialog scripts, the heart of many dialog systems, for the interactions with students. The scripts typically consist of a list of questions and corresponding ideal answers. In most ITSs, such scripts are manually crafted from instructional task descriptions. Such manual efforts not only cost more in terms of time and effort but also set a bottleneck in the scalability of the systems. Another major challenge they face is to automatically assess student answers with respect to the ideal answers. To address these challenges, this research proposes novel approaches to automatically generate questions. Furthermore, it focuses on finding appropriate approaches to assess and understand student responses in the form of natural text inputs. The question generation process generates cloze and open-cloze questions. Cloze questions are automatically generated by mining recorded tutorial dialogues between actual students and a state-of-the-art ITS. It complements the existing systems that rely only on the contents of instructional texts. Open-cloze questions are generated by minimizing human efforts. Specifically, active learning is used to train classifiers for judging the quality of automatically generated open-cloze questions, the most expensive step in generating open-cloze questions. Experiments show that a reasonably good classifier can be built with 300-500 examples labeled by using active learning which can provide about 5-10% more in accuracy and about 3-5% more in F1-measure than random sampling. Towards assessing and understanding student responses, this research addresses pronoun resolution and semantic textual similarity (STS) problems in the context of tutorial dialog. For pronoun resolution, a supervised machine learning approach is proposed which has a F-measure of 88.93%, showing its robustness in resolving pronouns. For assessing student responses, STS methods are used as they provide numeric scores indicating the degrees of equivalence in meaning between student answers and corresponding ideal answers. Since student responses in tutorial dialog are typically short in length, this research seeks to find the best methods for short text-to-text STS. To this end, Latent Dirichlet Allocation-based and regression-based methods are proposed. The methods are found to be very promising for computing short text-to-text semantic similarities. Although approaches to the STS problem provide numeric scores, they fail to explain the reasons behind them. To this direction, an interpretable STS system has been proposed which has been ranked at the top tier of this kind in the literature.

Comments

Data is provided by the student.

Library Comment

Dissertation or thesis originally submitted to the local University of Memphis Electronic Theses & dissertation (ETD) Repository.

Download

Share

COinS