A first evaluation of logic form identification systems
Abstract
This paper presents a first experience with evaluating sytems that address the issue of Logic Form Identification (LFi). A Gold Standard approach was used in which experts provide solutions to test data. The expert solutions, the gold standard, are then compared against outputs from participanting systems and different metrics observed. We proposed a few novel metrics, including precision and recall, that are further used to provide comparative results. The test data included 4155 arguments and 2398 predicates grouped in 300 sentences.
Publication Title
Proceedings of the SENSEVAL@ACL 2004: 3rd International Workshop on the Evaluation of Systems for the Semantic Analysis of Text - Held in cooperation with ACL 2004
Recommended Citation
Rus, V. (2004). A first evaluation of logic form identification systems. Proceedings of the SENSEVAL@ACL 2004: 3rd International Workshop on the Evaluation of Systems for the Semantic Analysis of Text - Held in cooperation with ACL 2004, 37-40. Retrieved from https://digitalcommons.memphis.edu/facpubs/2363
