The right threshold value: What is the right threshold of cosine measure when using latent semantic analysis for evaluating student answers?
Abstract
AutoTutor is an intelligent tutoring system that holds conversations with learners in natural language. AutoTutor uses Latent Semantic Analysis (LSA) to match student answers to a set of expected answers that would appear in a complete and correct response or which reflect common but incorrect understandings of the material. The correctness of student contributions is decided using a threshold value of the LSA cosine between the student answer and the expectations. In previous work LSA has shown to be effective in detecting good answers of students. The results indicate that the best agreement between LSA matches and the evaluations of subject matter experts can be obtained if the cosine threshold is allowed to be a function of the lengths of both student answer and the expectation being considered. Based on some of our experiences with LSA and AutoTutor, we are developing a new mathematical model to improve the precision of AutoTutor's natural language understanding and discriminative ability. © World Scientific Publishing Company.
Publication Title
International Journal on Artificial Intelligence Tools
Recommended Citation
Penumatsa, P., Ventura, M., Graesser, A., Louwerse, M., Hu, X., Cai, Z., & Franceschetti, D. (2006). The right threshold value: What is the right threshold of cosine measure when using latent semantic analysis for evaluating student answers?. International Journal on Artificial Intelligence Tools, 15 (5), 767-777. https://doi.org/10.1142/S021821300600293X
