Evaluation dataset (dt-grade) andwordweighting approach towards constructed short answers assessment in tutorial dialogue context
Abstract
Evaluating student answers often requires contextual information, such as previous utterances in conversational tutoring systems. For example, students use coreferences and write elliptical responses, i.e. incomplete but can be interpreted in context. The DT-Grade corpus which we present in this paper consists of short constructed answers extracted from tutorial dialogues between students and an Intelligent Tutoring System and annotated for their correctness in the given context and whether the contextual information was useful. The dataset contains 900 answers (of which about 25% required contextual information to properly interpret them). We also present a baseline system developed to predict the correctness label (such as correct, correct but incomplete) in which weights for the words are assigned based on context.
Publication Title
Proceedings of the 11th Workshop on Innovative Use of NLP for Building Educational Applications, BEA 2016 at the 2016 Conference of the North American Chapter of the Association for Computational Linguistics: Human Language Technologies, NAACL-HLT 2016
Recommended Citation
Banjade, R., Maharjan, N., Niraula, N., Gautam, D., Samei, B., & Rus, V. (2016). Evaluation dataset (dt-grade) andwordweighting approach towards constructed short answers assessment in tutorial dialogue context. Proceedings of the 11th Workshop on Innovative Use of NLP for Building Educational Applications, BEA 2016 at the 2016 Conference of the North American Chapter of the Association for Computational Linguistics: Human Language Technologies, NAACL-HLT 2016, 182-187. Retrieved from https://digitalcommons.memphis.edu/facpubs/2787
