A study of probabilistic and algebraic methods for semantic similarity


We study and propose in this article several novel solutions to the task of semantic similarity between two short texts. The proposed solutions are based on the probabilistic method of Latent Dirichlet Allocation (LDA) and on the algebraic method of Latent Semantic Analysis (LSA). Both methods, LDA and LSA, are completely automated methods used to discover latent topics or concepts from large collection of documents. We propose a novel word-to-word similarity measure based on LDA as well as several text-to-text similarity measures. We compare these measures with similar, known measures based on LSA. Experiments and results are presented on two data sets: the Microsoft Research Paraphrase corpus and the User Language Paraphrase corpus. We found that the novel word-to-word similarity measure based on LDA is extremely promising. Copyright © 2013, Association for the Advancement of Artificial Intelligence. All rights reserved.

Publication Title

FLAIRS 2013 - Proceedings of the 26th International Florida Artificial Intelligence Research Society Conference

This document is currently not available here.